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Abstract 

Though switched dynamical systems have shown great utility in modeling a variety of physical phe- 
nomena, the construction of an optimal control of such systems has proven difficult since it demands 
some type of optimal mode scheduling. In this paper, we devise an algorithm for the computation of 
an optimal control of constrained nonlinear switched dynamical systems. The control parameter for 
such systems include a continuous- valued input and discrete- valued input, where the latter corresponds 
to the mode of the switched system that is active at a particular instance in time. Our approach, 
which we prove converges to local minimizers of the constrained optimal control problem, first relaxes 
the discrete- valued input, then performs traditional optimal control, and then projects the constructed 
relaxed discrete-valued input back to a pure discrete-valued input by employing an extension to the 
classical Chattering Lemma that we prove. We extend this algorithm by formulating a computationally 
implementable algorithm which works by discretizing the time interval over which the switched dynami- 
cal system is defined. Importantly, we prove that this implementable algorithm constructs a sequence of 
points by recursive application that converge to the local minimizers of the original constrained optimal 
control problem. Four simulation experiments are included to validate the theoretical developments. 

1 Introduction 

Hybrid dynamical models arise naturally in systems in which discrete modes of operation interact with 
continuous state evolution. Such systems have been used in a variety of modeling applications including 
automobiles and locomotives employing different gears (20j [33] , biological systems [15] , situations where a 
control module has to switch its attention among a number of subsystems (23 [32] [39J , manufacturing systems 
[7] and situations where a control module has to collect data sequentially from a number of sensory sources 
[BJ [Hj ■ In addition, many complex nonlinear dynamical systems can be decomposed into simpler linear modes 
of operation that are more amenable to analysis and controller design [TH [TB] . 

Given their utility, there has been considerable interest in devising algorithms to perform optimal control 
of such systems. In fact, even Branicky et al.'s seminal work which presented many of the theoretical 
underpinnings of hybrid systems included a set of sufficient conditions for the optimal control of such systems 
using quasi variational inequalities [5]- Though compelling from a theoretical perspective, the application 
of this set of conditions to the construction of a numerical optimal control algorithm for hybrid dynamical 
systems requires the application of value iterations which is particularly difficult in the context of switched 
systems, wherein the switching between different discrete modes is specified by a discrete-valued input 
signal. The control parameter for such systems has both a discrete component corresponding to the schedule 
of discrete modes visited and two continuous components corresponding to the duration of time spent in 
each mode in the mode schedule and the continuous input. The determination of an optimal control for this 
class of hybrid systems is particularly challenging due to the combinatorial nature of calculating an optimal 
mode schedule. 

*R. Vasudevan, H. Gonzalez, R. Bajcsy, and S. S. Sastry are with the Department of Electrical Engineering and Computer 
Sciences, University of California at Berkeley, Berkeley, CA, 94720, {ramv,hgonzale,bajcsy,sastry,}@eecs. berkeley.edu 
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1.1 Related Work 



The algorithms to solve this switched system optimal control problem can be divided into two distinct groups 
according to whether they do or do not rely on the Maximum Principle [551 123 ISZ] ■ Given the difficulty of 
the problem, both groups of approaches sometimes employ similar tactics during algorithm construction. A 
popular such tactic is one formalized by Xu et al. who proposed a bi-level optimization scheme that at a 
low level optimized the continuous components of the problem while keeping the mode schedule fixed and 
at a high level modified the mode schedule [42] . 

We begin by describing the algorithms for switched system optimal control that rely on the Maximum 
Principle. One of the first such algorithms, presented by Alamir et al., applied the Maximum Principle 
directly to a discrete time switched dynamical system [T]. In order to construct such an algorithm for 
a continuous time switched dynamical system, Shaikh et al. employed the bi-level optimization scheme 
proposed by Xu et al. and applied the Maximum Principle to perform optimization at the lower level and 
applied the Hamming distance to compare different possible nearby mode schedules |35j . 

Given the algorithm that we construct in this paper, the most relevant of the approaches that rely on the 
Maximum Principle is the one proposed by Bcngca et al. who relax the discrete-valued input and treat it as 
a continuous- valued input over which they can apply the Maximum Principle to perform optimal control [3j . 
A search through all possible discrete valued inputs is required in order to find one that approximates the 
trajectory of the switched system due to the application of the constructed relaxed discrete-valued input. 
Though such a search is expensive, the existence of a discrete-valued input that approximates the behavior 
of the constructed relaxed discrete- valued input is proven by the Chattering Lemma [2|. Unfortunately 
this combinatorial search is unavoidable by employing the Chattering Lemma since it provides no means 
to construct a discrete-valued input that approximates a relaxed discrete-valued input with respect the 
trajectory of the switched system. Summarizing, those algorithms that rely on the Maximum Principle 
construct powerful necessary conditions for optimality. Unfortunately their numerical implementation for 
nonlinear switched systems is fundamentally restricted due to their reliance on approximating strong or 
needle variations with arbitrary precision as explained in 27 . 

Next, we describe the algorithms that do not rely on the Maximum Principle but rather employ weak 
variations. Several have focused on the optimization of autonomous switched dynamical systems (i.e. systems 
without a continuous input) by fixing the mode sequence and working on devising first [10] and second order 
[22j numerical optimal control algorithms to optimize the amount of time spent in each mode. In order to 
extend these optimization techniques, Axelsson et al. employed the bi-level optimization strategy proposed 
by Xu et al., and after performing optimization at the lower-level by employing a first order numerical optimal 
control algorithm to optimize the amount of time spent in each mode while keeping the mode schedule fixed, 
they modified the mode sequence by employing a single mode insertion technique [2]. 

There have been two major extensions to Axelsson et al.'s algorithm. First, Wardi et al., extend the 
approach by performing several single mode insertions at each iteration [40 . Second, Gonzalez et al., 
extend the approach to make it applicable to constrained switched dynamical systems with a continuous- 
valued input [T7] |TS] . Though these single mode insertion techniques avoid the computational expense of 
considering all possible mode schedules during the high-level optimization, this improvement comes at the 
expense of restricting the possible modifications of the existing mode schedule, which may introduce undue 
local minimizers, and at the expense of requiring a separate optimization for each of the potential mode 
schedule modifications, which is time consuming. 

1.2 Our Contribution and Organization 

Inspired by the potential of the Chattering Lemma, in this paper, we devise and implement a first order 
numerical optimal control algorithm for the optimal control of constrained nonlinear switched systems. In 
Section [2] we introduce the notation and assumptions used throughout the paper and formalize the the 
optimal control for constrained nonlinear switched systems. Our approach to solve this problem, which is 
formulated in Section [3] first relaxes the optimal control problem by treating the discrete- valued input to be 
continuous- valued. Next, a first order numerical optimal control algorithm is devised for this relaxed problem. 
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After this optimization is complete, an extension of the Chattering Lemma that we construct, allows us to 
design a projection that takes the computed relaxed discrete-valued input back to a "pure" discrete-valued 
input while controlling the quality of approximation of the trajectory of the switched dynamical system 
generated by applying the projected discrete-valued input rather than the relaxed discrete-valued input. 
In Section [4j we prove that the sequence of points generated by recursive application of our first order 
numerical optimal control algorithm converge to a point that satisfies a necessary condition for optimality 
of the constrained nonlinear switched system optimal control problem. 

We then describe in Section [5] how our algorithm can be formulated in order to make numerical imple- 
mentation feasible. In fact, in Section [6j we prove that the this computationally implementable algorithm 
is a consistent approximation of our original algorithm. This ensures that the sequence of points generated 
by the recursive application of this numerically implementable algorithm converge to a point that satisfies 
a necessary condition for optimality of the constrained nonlinear switched system optimal control problem. 
In Section [7J we implement this algorithm and compare its performance to a commercial mixed integer 
optimization algorithm on 4 separate problems to illustrate its superior performance with respect to speed 
and quality of constructed minimizer. 



2 Preliminaries 

In this section, we formalize the problem we solve in this paper. Before describing this problem, we define 
the function spaces and norms used throughout this paper. 



2.1 Norms and Functional Spaces 

This paper focuses on the optimization of functions with finite L 2 -norm and finite bounded variation. To 
formalize this notion, we require a norm. For each x E R", peN, and p > 0, we let ||x|| denote the p-norm 
of x. For each A £ ]R™ xm , pgN, and p > 0, we let || denote the induced p-norm of A. 

Given these definitions, we say a function, / : [0, 1] —> y, where y C E™, belongs to L 2 ([0, l],y) with 
respect to the Lebesgue measure on [0, 1] if: 

ll/IL= = (£ll/(«)ll2^) 2 <°°- (1) 

We say a function, / : [0, 1] — > y, where JcM", belongs to L°°([0, with respect to the Lebesgue 
measure on [0, 1] if: 

\\f\\ L oc = inf{a > | ||/(a;)|| 2 < a for almost every x £ [0, 1]} < oo. (2) 

In order to define the space of functions of finite bounded variation, we first define the total variation of 
a function. Given P, the set of all finite partitions of [0, 1], we define the total variation of / : [0, 1] — > y by: 

Wfhv = I £ H/fe+i) - /feOlli I e p\ ■ (3) 

Note that the total variation of / is not a norm but rather a seminorm, i.e. it does not separate points. 
Regardless, we use the norm symbol for the total variation throughout this paper. We say that / is of 
bounded variation if < oo, and we define BV([0,l],y) to be the set of all functions of bounded 

variation from [0, 1] to y. 

There is an important connection between the functions of bounded variation and weak derivatives, which 
we rely on throughout this paper. Given / : [0, 1] — > y, we say that / has a weak derivative if there exists a 
Radon signed measure \x over [0, 1] such that, for each smooth bounded function v with v(0) = v(l) = 0, 

f(t)v(t)dt = - f v(t)dn{t). (4) 
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Moreover, we say that / = , where the derivative is taken in the Radon-Nikodym sense, is the weak 
derivative of /. Note that / is in general a distribution. Perhaps the most common example of weak 
derivative is the Dirac Delta, which is the weak derivative of the Step Function. The following result is 
fundamental in our analysis of functions of bounded variation: 

Theorem 1 (Exercise 5.1 in [13]). If f & BV([0, l],y), then f has a weak derivative, denoted f. Moreover, 

\\f\\BV= f \\mWxdt. (5) 
Jo 

We omit the proof of this result since it is beyond the scope of this paper. More details about the 
functions of bounded variation and weak derivatives can be found in Sections 3.5 and 9 in |13j and Section 
5 in 03]. 



2.2 Optimization Spaces 

We are interested in the control of systems whose trajectory is governed by a set of vector fields / : M x R" x 
M. m x Q — > R. n , indexed by their last argument where Q = {1,2, . . . , q}. Each of these distinct vector fields 
is called a mode of the switched system. To formalize the optimal control problem, we define three spaces: 
the pure discrete input space, T> p , the relaxed discrete input space, T> r , and the continuous input space, U. 
Throughout the document, we employ the following convention: given the pure or relaxed discrete input d, 
we denote its i-th coordinate by di. 

Before formally defining each space, we require some notation. Let the g-simplex, be defined as: 



S? = |(di,...,d fl )G[0,l]«|X;di = V ) (6) 
and let the corners of the g-simplex, E|, be defined as: 

£|= {(*,...,<*,) €{0,1}" >> = ] \- (7) 




Note that C SJ. Also, there are exactly as many corners, denoted ej for % E Q, of the g-simplex as there 
are distinct vector fields. Thus, T, p = {e\, . . . , e q }. 

Using this notation, we define the pure discrete input space, T> p , as: 

2? p = L 2 ([0,l],S«)nBV([0,l],S|). (8) 

Next, we define the relaxed discrete input space, T> r : 

P r -i 2 ([0,l],E«)nW([0,l],E«). (9) 

Notice that the discrete input at each instance in time can be written as the linear combination of the corners 
of the simplex. Given this observation, we employ these corners to index the vector fields (i.e. for each i e Q 
we write f {■,-,-, e^) for f (■,-,-, i)). Finally, we define the continuous input space, U: 

U = L 2 ([0,l},U)nBV([0,l},U), (10) 

where U C M." 1 is a bounded, convex set. 

Let X = L°°([Q, l],R m ) x L°°([0, l],R q ) be endowed with the following norm for each £ = (u,d) e X: 

\\t\\ X = \\u\\ L , + \\d\\ L *, (11) 

where the L 2 -norm is as defined in Equation ([TJ). We combine U and D p to define our pure optimization 
space, X p = IA x T> p , and we endow it with the same norm as X . Similarly, we combine U and T> r to define 
our relaxed optimization space, X r = hi x T> r , and endow it with the A'-norm too. Note that X p C X r C X . 
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2.3 Trajectories, Cost, Constraint, and the Optimal Control Problem 



Given £ = (u,d) G X r , for convenience throughout the paper we let: 

i 

f{t,x(t),u(t),d{t)) =J2<li(t)f{t,x(t),u{t),e i ), (12) 
i=i 

where d(t) = Yli=i ^»(*) e »- We employ the same convention when we consider the partial derivatives of /. 
Given Xo G R ra , we say that a trajectory of the system corresponding to £ G X r is the solution to: 

x{t)=f(t,x(t),u(t),d(t)), Vie [0,1], x(0)=x , (13) 

and denote it by x^ : [0, 1] -> E", where we suppress the dependence on xo in since it is assumed given. 
To ensure the clarity of the ensuing analysis, it is useful to sometimes emphasize the dependence of x^(t) 
on £. Therefore, we define the flow of the system, <\> t : X r — > M. n for each t G [0, 1] as: 

M0=x {i) (t). (14) 

To define the cost function, we assume that we are given a terminal cost, ho : K™ — > K. The cost function, 
J : X r — > K., for the optimal control problem is then defined as: 

J(0 = h (x^(l)). (15) 

Notice that if the problem formulation includes a running cost, then one can extend the existing state vector 
by introducing a new state, and modifying the cost function to evaluate this new state at the final time, 
as shown in Section 4.1.2 in [22]. By performing this type of modification, observe that each mode of the 
switched system can have a different running cost associated with it. 

Next, we define a family of functions, hj : R™ — > K for j G J = {1, . . . , N c }. Given a £ G X r , the state 
x^> is said to satisfy the constraint if hj(x^(t)) < for each t G [0, 1] and for each j G J . We compactly 
describe all the constraints by defining the constraint function "J/ : X r — > K, by: 

#(£)= max hJx^tt)), (16) 
jej, te[o,i] JK K " 

since hj(x^{t)^ < for each t and j if and only if ^(£) < 0. To ensure the clarity of the ensuing analysis, 
it is useful to sometimes emphasize the dependence of hj(x^(t)^) on £. Therefore, we define component 
constraint functions, ipjj : X r — >■ K for each t G [0, 1] and j G J as: 

With these definitions, we can state the Switched System Optimal Control Problem: 
Switched System Optimal Control Problem. 

min { J(£) | < 0} . (18) 



2.4 Assumptions and Uniqueness 

In order to devise an algorithm to solve Switched System Optimal Control Problem, we make the following 
assumptions about the dynamics, cost, and constraints: 

Assumption 1. For each i G Q, /(•,-, -,ej) is differentiate in both x and u. Also, each /(-,-, -,ej) and its 
partial derivatives are Lipschitz continuous with constant L > 0, i.e. given t\,ti G [0,1], X\,Xi G M™, and 
ui,u 2 G U: 
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(1) \\f(ti,xuui,ei) - f(t 2 ,x 2 ,u 2 ,e l )\\ 2 < L (\ti - t 2 | + \\xi - x 2 \\ 2 + \\ui - u 2 \\ 2 ), 



(2) ^(t 1 ,x 1 ,u 1 ,e l ) - ^(t 2 ,x 2 ,u 2 ,ei) 



Of, 



i,2 



<£(|*i-*a| 



\Xl - X 2 \\2 + \\ui - U2W2), 



( 3 ) %(h,xi,Ui,ei) - %L(t2,X2,U2,ei) < L (|t x - 1 2 | + \\xi - X2W2 + IK - ^Ha)- 



»,2 

Assumption 2. 77ie functions ho and hj are Lipschitz continuous and differentiable in x for all j € J7. 
In addition, the derivatives of these functions with respect to x are also Lipschitz continuous with constant 
L > 0, i.e. given x\,x 2 € K n , /or eacft j € J'-' 

(1) \h (xi) - h (x 2 )\ < L\\xx - x 2 \\ 2 , 

(2) ||t^i)-f?M| 2 <L|ki-x 2 || 27 

(3) - hj(x 2 )\ < L \\xi - x 2 \\ 2 , 



(4) 



< L Marx - £2! 



If a running cost is included in the problem statement (i.e. if the cost also depends on the integral of a 
function), then this function must also satisfy Assumption [I] Assumption [2] is a standard assumption on the 
objectives and constraints and is used to prove the convergence properties of the algorithm defined in the 
next section. These assumptions lead to the following result: 



Lemma 1. There exists a constant C > such that, for each £ £ X r and t £ [0, 1], 

||x< e >(i)|| 2 <C, 
where x^ is a solution of Differential Equation (13 I. 

Proof. Given £ = (it, d) £ X r and noticing that |d,:(i)| < 1 for all i £ Q and t £ [0, 1], we have: 



(19) 



X^\t)\L<\\x \\2 + 



1 ft 

y~] / \\f(s,x ii) {s),u(s),e l )\\ 2 ds. 



(20) 



Next, observe that ||/(0, Xq, 0, ej)|| 2 is bounded for all i £ Q and u(s) is bounded for each s £ [0, 1] since 
U is bounded. Then by Assumption [TJ we know there exists a K > such that for each s £ [0, 1], i £ Q, 
and £ £ X ri 

\\f(s,x^(s),u(s),e i )\\ 2 <K(\\x^(s)\\ 2 + l). (21) 

Applying the Bellman-Gronwall Inequality (Lemma 5.6.4 in to Equation (20), we have ||x^(t)|| 2 < 
e qK (l + ||xo|| 2 ) for each t £ [0, 1]. Since x is assumed given and bounded, we have our result. □ 

In fact, this implies that the dynamics, cost, constraints, and their derivatives are all bounded: 

Corollary 1. There exists a constant C > such that for each £ = (u, d) £ X r , t £ [0, 1], and j £ J : 



(1) \\f(t,x^(t),u(t),d(t))\\ 2 <c, |£M«(*)> «(*),<*(*)) 



<c, 



li(t,x^ { t),u(t),d(t)) 



<C, 



(2) I fto (!«)(*)) I <C, 

(3) |^(x(«)(t))|<C, 



^(*«>(t)) 



dx 



(*)) 



where x^' is a solution of Differential Equation (13 I. 



G 



Proof. The result follows immediately from the continuity of /, ho, ^f, /ij, and -g^f for each 

j £ i7, as stated in Assumptions [I] and [2] and the fact that each of the arguments to these functions can be 
constrained to a compact domain, which follows from Lemma [l] and the compactness of U and □ 

An application of this corollary leads to a fundamental result: 



Theorem 2. For each £ £ X r Differential Equation (13) has a unique solution 



Proof. First let us note that /, as defined in Equation (12), is also Lipschitz with respect to its fourth 



argument. Indeed, given t £ [0, 1], x £ K™, u £ U, and d\, d% £ 



|/(i,x,M,<ii) - /(*, a;, tt, d 2 ) I 
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E 

i=l 



(diA - d 2tl )f(t,x,u,ei) 



(22) 



<C9||di-d 2 ||2, 



where C > is as in Corollary [T] 

Given that / is Lipschitz with respect to all its arguments, the result follows as a direct extension of the 
classical existence and uniqueness theorem for nonlinear differential equations (see Section 2.4.1 in [35] for a 
standard version of this theorem) . □ 

Therefore, since x^) is unique, it is not an abuse of notation to denote the solution of Differential Equation 



(13) by x^K Next, we develop an algorithm to solve the Switched System Optimal Control Problem. 



3 Optimization Algorithm 

In this section, we describe our optimization algorithm. Our approach proceeds as follows: first, we treat a 
given pure discrete input as a relaxed discrete input by allowing it to belong 2? r ; second, we perform optimal 
control over the relaxed optimization space; and finally, we project the computed relaxed input into a pure 
input. Before describing our algorithm in detail, we begin with a brief digression to motivate why such a 
roundabout construction is required in order to devise a first order numerical optimal control scheme for the 



Switched System Optimal Control Problem defined in Equation (18). 



3.1 Directional Derivatives 

To appreciate why the construction of a numerical scheme to find the local minima of the Switched System 



Optimal Control Problem defined in Equation (18) is difficult, suppose that the optimization in the problem 
took place over the relaxed optimization space rather than the pure optimization space. The Relaxed 
Switched System Optimal Control Problem is then defined as: 

Relaxed Switched System Optimal Control Problem. 

min { J(0 | *(0 < 0} . (23) 

The local minimizers of this problem are then defined as follows: 

Definition 1. Let us denote an e-ball in the X-norm centered at £ by: 

M x {(,,e) = {^X r \ \\£-t\\ x<e }. (24) 

We say that a point £ € X r is a local minimizer of the Relaxed Switched System Optimal Control Problem 
defined in Equation ( [23] ) £/*&(£) < and there exists e > such that J(£) > J(£) for each £ £ A0t(£,£) D 
{£"eAV|*(0<0}. 



7 



Given this definition, a first order numerical optimal control scheme can exploit the vector space structure 
of the relaxed optimization space in order to define directional derivatives that find local minimizers for this 
Relaxed Switched System Optimal Control Problem. 

To concretize how such an algorithm would work, we introduce some additional notation. Given £ G X r , 
y a Euclidean space, and any function G : X r — > y, the directional derivative of G at £, denoted DG(£; •) : 
X — > y, is computed as: 

re(ftO-Hm g « + A ?- g <fl (25) 
no A 

To understand the connection between directional derivatives and local minimizers, suppose the Relaxed 
Switched System Optimal Control Problem is unconstrained and consider the first order approximation of 
the cost J at a point £ £ X r in the £' £ X direction by employing the directional derivative DJ(£;£'): 

J(£ + A£')« J(£) + ADJ(£;£'), ( 26 ) 

where < A <C 1. It follows that if DJ(£;£'), whose existence is proven in Lemma[8j is negative, then it is 
possible to decrease the cost by moving in the £' direction. That is if the directional derivative of the cost 
at a point £ is negative along a certain direction, then for each e > there exists a £ € A/^(£, s) such that 
J(£) < J(£). Therefore if D J(£; £') is negative, then £ is not a local minimizer of the unconstrained Relaxed 
Switched System Optimal Control Problem. 

Similarly, for the general Relaxed Switched System Optimal Control Problem, consider the first order 
approximation of each of the component constraint functions, ipjj for each j £ J and t £ [0, 1] at a point 
£ £ X r in the £ £ A" direction by employing the directional derivative D?/)j it (£; £'): 

lM£ + AO « <M£) + AD^- 1 (£;£'), (27) 

where < A <C 1. It follows that if D^ t (£; £'), whose existence is proven in Lemma [9j is negative, then it is 
possible to decrease the infeasibility of <^>t(£) with respect to hj by moving in the £' direction. That is if the 
directional derivatives of the cost and all of the component constraints for all t £ [0, 1] at a point £ are negative 
along a certain direction and , 3 / (£) = 0, then for each e > there exists a £ £ {£ £ X r | \l/(£) < 0} n A/";t (£, e) 
such that J(£) < J(£). Therefore, if vfr(£) = and DJ(£;£') and D^,t(£;£') are negative for all j £ J and 
t £ [0,1], then £ is not a local minimizer of the Relaxed Hybrid Optimal Control Problem. Similarly, if 
'!'(£) < and DJ(£;£') is negative, then £ is not a local minimizer of the Relaxed Hybrid Optimal Control 
Problem, even if D^j it (£; £') is greater than zero for all j £ J and t £ [0, 1]. 

Returning to the Switched System Optimal Control Problem, it is unclear how to define a directional 
derivative for the pure discrete input space since it is not a vector space. Therefore, in contrast to the relaxed 
discrete and continuous input spaces, the construction of a first order numerical scheme for the optimization 
of the pure discrete input is non-trivial. One could imagine trying to exploit the directional derivatives in 
the relaxed optimization space in order to construct a first order numerical optimal control algorithm for 
the Switched System Optimal Control Problem, but this would require devising some type of connection 
between points belonging to the pure and relaxed optimization spaces. 



3.2 The Weak Topology on the Optimization Space and Local Minimizers 

To motivate the type of relationship required between the pure and relaxed optimization space in order to 
construct a first order numerical optimal control scheme, we begin by describing the Chattering Lemma: 

Theorem 3 (Theorem 1 in [3 ]). For each £ r £ X r and e > there exists a £ p £ X p such that for each 
t £ [0,1]: 

||<M£r) - <M£p)|| 2 < e, (28) 



where ^t(£ r ) and <^»t(£p) o r e solutions to Differential Equation (13) corresponding to £ r and £ p , respectively. 
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The theorem as is proven in [3] is not immediately applicable to switched systems, but a straightforward 
extension as is proven in Theorem 1 in 3J makes that feasible. Note that the theorem as stated in [3J, 
considers only two vector fields (i.e. q = 2), but as the author's of the theorem remark, their proof can be 
generalized to an arbitrary number of vector fields. A particular version of this existence theorem can also 
be found in Lemma 1 |36| . 

Theorem [3] says that the behavior of any element of the relaxed optimization space with respect to the 
trajectory of switched system can be approximated arbitrarily well by a point in the pure optimization space. 
Unfortunately, the relaxed and pure point as in Theorem[3]need not be near one another in the metric induced 
by the A'-norm. Therefore, though there exists a relationship between the pure and relaxed optimization 
spaces, this connection is not reflected in the topology induced by the X-norm; however, in a particular 
topology over the relaxed optimization space, a relaxed point and the pure point that approximates it as in 
Theorem [3] can be made arbitrarily close: 



Definition 2. We say that the weak topology on X r induced by Differential Equation (13) is the smallest 
topology on X r such that the map £ 1— > x^ is continuous. Moreover, an e-ball in the weak topology centered 
at £ is denoted by: 

K,(t, e) = {f G X r I - x® \\ L2 < e} . (29) 

A longer introduction to weak topology can be found in Section 3.8 in [31] or Section 2.3 in [21], but 
before continuing we make an important observation that aids in motivating the ensuing analysis. In order 
to understand the relationship between the topology generated by the A"-norm on X r and the weak topology 
on X r , observe that (f> t is Lipschitz continuous for all t € [0, 1] (this is proven in Corollary [3]). Therefore, for 
any e > there exists a 5 > such that if a pair of points of the relaxed optimization space belong to the 
same 5— ball in the A-norm, then the pair of points belong to the same e-ball in the weak topology on X r . 

Notice, however, that it is not possible to show that for every s > that there exists a 5 > such that if 
a pair of points of the relaxed optimization space belong to the same (S-ball in the weak topology on X r , then 
the pair of points belong to the same e-ball in the X -norm. More informally, a pair of points may generate 
trajectories that are near one another in the L 2 -norm while not being near one another in the X-norm. 
Since the weak topology, in contrast to the A-norm induced topology, naturally places points that generate 
nearby trajectories next to one another, we extend Definition [2] in order to define a weak topology on X p 
which we then use to define a notion of local minimizer for the Switched System Optimal Control Problem: 

Definition 3. We say that a point £ £ X p is a local minimizers of the Switched System Optimal Control 
Problem defined in Equation ( |18[ ) if ^(£) < and there exists e > such that </(£) > </(£) for each 
£ € N" w (£,e) H {£ € X p | < 0}, where N w is as defined in Equation ( [29] ). 

With this definition of local minimizer, we can exploit Theorem [3j even just as an existence result, 
along with the notion of directional derivative over the relaxed optimization space to construct a necessary 
condition for optimality for the Switched System Optimal Control Problem. 

3.3 An Optimality Condition 

Motivated by the approach undertaken in [29], we define an optimality function, 9 : X p — > (— 00, 0] that 
determines whether a given point is a local minimizer of the Switched System Optimal Control Problem and 
a corresponding descent direction, g : X v X r : 



= mm C&O. 5(0 = argminC(0O, (30) 



where 



max DJ(£;0-0, max W jtt ({; ? - + 7*(0 \ + W ~ £11* tf*(0<0, 

au') = { \ ^ te[a11 J (3i) 

max DJ(£;0 -0-*(0» ma ^ ^AtC -0\ + \W -CWx if > 0, 
[ jeJ, te[o,ij J 
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where 7 > is a design parameters. For notational convenience in the previous equation we have left out the 
natural inclusion of £ from X p to X r . Before proceeding, we make two observations. First, note that 6(d) < 
for each £ X p , since we can always choose = £ which leaves the trajectory unmodified. Second, note that 
at a point £ £ X p the directional derivatives in the optimality function consider directions £' — £ with £' G A",, 
in order to ensure that first order approximations constructed as in Equations ( 26 ) and ( 27 1 belong to the 
relaxed optimization space X r which is convex (e.g. for < A <C 1, J(d) + AD J(£; £' — £) ~ J((l — A)£ + A£') 
where (1 — A)£ + A£' G X r ). 

To understand how the optimality function behaves, consider several cases. First, if 6(d) < and 
^(d) = 0, then there exists a £' G AV such that both DJ(£;£' — £) and D^, — £) are negative for all 
j G i7 and f G [0,1]. By employing the aforementioned first order approximation, we can show that for each 
e > there exists an e-ball in the A-norm centered at £ such that J(£) < J(£) for some £ G {£ G X r | 
^(C) < 0} n A^f (£, £■)• As a result and because the cost and each of the component constraint functions are 
assumed Lipschitz continuous and <j>t for all t G [0, 1] is Lipschitz continuous as is proven in Corollary [3j an 
application of Theorem [3] allows us to show that for each e > there exists an e-ball in the weak topology on 
X p centered at £ such that J(d p ) < J(£) for some dp G {£ G A" p | *(£) < 0} nAt((,4 Therefore, it follows 
that if 6(d) < and , 5(£) = 0, then £ is not a local minimizer of the Switched System Optimal Control 
Problem. 

Second, if 6(d) < and 4 r (£) < 0, then there exists a G X r such that DJ(£; £' — d) is negative. Though 
Dtl)j i t(d;d' ~ d) maybe positive for some j G J and t G [0, 1], by employing the aforementioned first order 
approximation, we can show that for each e > there exists an e-ball in the X-norm centered at £ such that 
J(d) < J(d) for some £ G {£ G X r | ^(£) < 0} n Mx(di £ )- As a result and because the cost and each of the 
constraint functions are assumed Lipschitz continuous and <j>t for all t G [0,1] is Lipschitz continuous as is 
proven in Corollary[3j an application of Theorem [3] allows us to show that for each e > there exists an e-ball 
in the weak topology on X p centered at £ such that J(dp) < J(d) f° r some dp G {£ G X p | ^>(d) < 0}P\Af w (d, e). 
Therefore, it follows that if 9(d) < and &(d) < 0, then £ is not a local minimizer of the Switched System 
Optimal Control Problem. In this case, the addition of the 4" term in £ ensures that a direction that reduces 
the cost does not simultaneously require a decrease in the infeasibility in order to be considered as a potential 
descent direction. 

Third, if 9(d) < and ^f(d) > 0, then there exists a £' G X r such that T)tpj t (d;d' ~ d) 1S negative for 
all 3^3 and t G [0, 1]. By employing the aforementioned first order approximation, we can show for each 
e > there exists an e-ball in the A-norm centered at £ such that v f(£) < ^(d) for some d G Nx(di £ )- As a 
result and because each of the constraint functions are assumed Lipschitz continuous and 4>t for all t G [0, 1] 
is Lipschitz continuous as is proven in Corollary [3j an application of Theorem [3] allows us to show that for 
each e > there exists an e-ball in the weak topology on X p centered at £ such that < ^(d) f° r some 

dp G J\f w (d,s). Therefore, though it is clear that £ is not a local minimizer of the Switched System Optimal 
Control Problem since ^(d) > 0, it follows that if 6(d) < and *&(£) > 0, then it is possible to locally reduce 
the infeasibility of £. In this case, the addition of the D J term in £ serves as a heuristic to ensure that the 
reduction in infeasibility does not come at the price of an undue increase in the cost. 

These observations are formalized in Theorem [5] where we prove that if £ is a local minimizer of the 
Switched System Optimal Control Problem, then 6(d) — 0, or that 6(d) = is a necessary condition for the 
optimality of £. To illustrate the importance of 9 satisfying this property, recall how the directional derivative 
of a cost function is employed during unconstrained finite dimensional optimization. Since the directional 
derivative of the cost function at a point being equal to zero in all directions is a necessary condition for 
optimality for an unconstrained finite dimensional optimization problem, it is used as a stopping criterion by 
first order numerical algorithms (Corollary 1.1.3 and Algorithm Model 1.2.23 in [29]). Similarly, by satisfying 
Theorem [5j 6 is a necessary condition for optimality for the Switched System Optimal Control Problem and 
can therefore be used as a stopping criterion for a first order numerical optimal control algorithm trying to 
solve the Switched System Optimal Control Problem. Given (9's importance, we say a point, d G X p , satisfies 
the optimality condition if 6(d) = 0. 
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3.4 Choosing a Step Size and Projecting the Relaxed Discrete Input 

Impressively, Theorem[3]just as an existence result is sufficient to allow for the construction of an optimality 
function that encapsulates a necessary condition for optimality for the Switched System Optimal Control 
Problem. Unfortunately, Theorem [3] is unable to describe how to exploit the descent direction, <?(£), since its 
proof provides no means to construct a pure input that approximates the behavior of a relaxed input while 
controlling the quality of the approximation. In this paper, we extend Theorem [3] by devising a scheme that 
remedies this shortcoming. This allows for the development of a numerical optimal control algorithm for the 
Switched System Optimal Control Problem that first, performs optimal control over the relaxed optimization 
space and then projects the computed relaxed control into a pure control. 

Before describing the construction of this projection, we describe how the descent direction, (?(£), can 
be exploited to construct a point in the relaxed optimization space that either reduces the cost (if the £ is 
feasible) or the infeasibility (if £ is infeasibile) . Comparing our approach to finite dimensional optimization, 
the argument that minimizes C is a "direction" along which to move the inputs in order to reduce the cost in 
the relaxed optimization space, but we require an algorithm to choose a step size. We employ a line search 
algorithm similar to the traditional Armijo algorithm used during finite dimensional optimization in order 
to choose a step size (Algorithm Model 1.2.23 in [19]). Fixing a £ (0, 1) and j3 £ (0, 1), a step size for a 
point £ £ X p is chosen by solving the following optimization problem: 

fmin{fc £ N | J(£ + /3 fc (<?(£) - 0) ~ AO < <*0 k O(Q, 
m = \ *(e + /? fe (.9(0-0) <u[3 k 8(0} if *(0 < 0, (32) 

[min{fc £ N | + /3 fe (.g(£) - £)) - *(0 < a/3 fc 0(£)} if 



> 0. 



In Lemma 
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we prove that for £ £ X p , if 6(£) < 0, then /i(£) < oo. Therefore, if < for some £ £ X p , 
then we can construct a descent direction, <?(£), and a step size, fi(£,), and a new point (£ + ft^^ (<?(£) — £)) £ 
X r that produces a reduction in the cost (if £ is feasible) or a reduction in the infeasibility (if £ is infeasible) . 

We define the projection that takes this constructed point to a point belonging the pure optimization 
space while controlling the quality of approximation in two steps. First, we approximate the relaxed input by 
its 7V-th partial sum approximation via the Haar wavelet basis. To define this operation, Tjy : L 2 ([0, 1], R) n 
BV([0, 1],R) -> L 2 ([0, 1],R) n BV([0, 1],R), we employ the Haar wavelet (Section 7.2.2 in [56]): 

(l iftG[0,i), 
A(t)=|-1 ifte [|,1), (33) 
[0 otherwise. 

Letting 1 : R —> R be the constant function equal to one and : [0, 1] — > R for k £ N and jg{0,..., 2 k — 1}, 
be defined as b kj (t) = \(2 k t-j), the projection F N for some c £ L 2 ([0, 1], R) n BV([0, 1], R) i 2 ([0,l],R)n 
W([0,1],R) is defined as: 

2^ 1 

[T N (c)](t) - (el) +E E ( c '^) nT#- (34) 

k=0 3=0 " k 3"L* 

Note that the inner product here is the traditional Hilbert space inner product. 

This projection is then applied to each of the coordinates of an element in the relaxed optimization space. 
To avoid introducing additional notation, we let the coordinate-wise application of J-jv to some relaxed 
discrete input d £ T> r be denoted as Fn(cI) and similarly for some continuous input u £lA. Lemma 
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proves 



that for each N £ N, each t £ [0, 1], and each i £ {1, . . . , q}, [Tn^)^ (t) £ [0, 1] and £? =1 [J 7 N (d)] t (t) = 1 
for the projection Jjy(d). Therefore it follows that for each d £ T> r , Jjv(^) S ^V- 

Second, we project the output of ^^(d) to a pure discrete input by employing the function Vn ■ 2-V — > D p 
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which computes the pulse width modulation of its argument with frequency 2 



-N. 



if t G 



2- N (k + Efcx 4* (£)) - (* + Ej=i (w))) , G {0, 1, . . . , 2^ - 1} 



otherwise. 



, (35) 
Lemma 13 proves that for each ffeN, each f G [0, 1], and each i G {1, . . . , q}, [V N (T N (d))] (t) G {0, 1} 
and J2i=i ^ 3 Jv(-7 r A'(rf))] i (i) = 1- This proves that r P N (^J r N (d)) G V p for each d G X> r . 



Fixing ]V e N, we compose the two projections and define pn '■ X, 

p N (u,d) = (j r A r(u),'Pjv(-7 : "A'(rf)) 



(36) 



Critically, as shown in Theorem [6j this projection allows us to extend Theorem [3] by constructing an upper 
bound that goes to zero as N goes infinity between the error of employing the relaxed control rather than its 
projection in the solution of Differential Equation ( 13 ). Therefore in a fashion similar to applying the Armijo 
algorithm, we choose an N G N at which to perform pulse width modulation by performing a line search. 
Fixing q G (0, oo), (3 G (^^i lj > an d u G (0, 1), a frequency at which to perform pulse width modulation for 
a point £ G X p is computed by solving the following optimization problem: 



In Lemma 



min{fc G N | J(p fc (£ + /^%(£) - £))) - AO < - ^ k )0(O, 

*(p fe (£ + /3 M( %(£)-£))) <0, a/3 fc < (l-w)a^)} if < 0, 

minjA; G N | *( p ,(£ + ^(«( 5 (£) - £))) - *(£) < (a/3"«) - a^ fe )0(£), 

a/3 fe < (1 - w)a/3^ (?) } if *(£) > 0. 



(37) 
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we prove that for £ G A" p , if #(£) < 0, then i/(£) < oo. Therefore, if 6*(£) < for some £ G X p , 
then we can construct a descent direction, <?(£), a step size, /x(£), a frequency at which to perform pulse 
width modulation, v (£), and a new point p v (p\\t + (<?(£) — £)) G X p that produces a reduction in the 
cost (if £ is feasible) or a reduction in the infeasibility (if £ is infeasible) . 



3.5 Switched System Optimal Control Algorithm 

Consolidating our definitions, Algorithm [T] describes our numerical method to solve the Switched System 
Optimal Control Problem. For analysis purposes, we define r : X p —¥ X p by 

r(O = P^)(£ + 0" ( %(O-O). (38) 

We say {£j}jeN is a sequence generated by Algorithm [7] if £ J+1 = r(£ J ) for each j G N. We can prove several 
important properties about the sequence generated by Algorithm [T] First, in Lemma |18[ we prove that if 
there exists io G N such that ^(£i ) < 0, then , 5(£i) < for each i > iq. That is, if the Algorithm constructs 
a feasible point, then the sequence of points generated after this feasible point are always feasible. Second, in 
Theorem|9j we prove lim :) _ i . 00 #(£_,) = or that Algorithm [l] converges to a point that satisfies the optimality 
condition. 



4 Algorithm Analysis 

In this section, we derive the various components of Algorithm [T] and prove that Algorithm [T] converges 
to a point that satisfies our optimality condition. Our argument proceeds as follows: first, we prove the 
continuity of the state, cost, and constraint, which we employ in latter arguments; second, we construct 
the components of the optimality function and prove that these components satisfy various properties that 
ensure that the well-posedness of the optimality function; third, we prove that we can control the quality of 
approximation between the trajectories generated by a relaxed discrete input and its projection by pjq as a 
function of N; finally, we prove the convergence of our algorithm. 
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Algorithm 1 Optimization Algorithm for the Switched System Optimal Control Problem 
Require: £o e X p , a e (0, 1), a G (0, oo), /3 € (0, 1), /3 € l) , 7 G (0, 00), w G (0, 1). 
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Set j = 0. 

Compute as defined in Equation (30) 
if e(lj) = then 

return £j. 
end if 



Compute g(£j) as defined in Equation (30). 
Compute /i(£j) as defined in Equation (32) 



Compute as defined in Equation (37). 

Set = + P^Hdi^j) ~ £j)7' as defined in Equation (36) 

Replace j by j + 1 and go to Line [5J 



4.1 Continuity 

In this subsection, we prove the continuity of the state, cost, and constraint. We begin by proving the 



continuity of the solution to Differential Equation ( 13 1 with respect to £ by proving that this mapping is 
sequentially continuous: 

Lemma 2. Let C X r be a convergent sequence with limit £ G X r . Then the corresponding sequence 

of trajectories {x^ j ^}j° =1 , as defined in Equation (13), converges uniformly to x^K 



Proof. For notational convenience, let = (uj, dj), £ = (it, d), and <p t as defined in Equation (14). We begin 
by proving the convergence of {</>*(£) to <M£) for each t 6 [0, 1]. Consider 



\\Mtj)-M0\\ 2 = 

Therefore, 

\\MZj)-M£)\\2 = 



t 1 



(39) 



t 1 



£ (fc-Mr) - (r))/(r, r & ) , Uj (r) , e,) + 
»=i 

+ ^ ( T ) (/ ( T , (£j ) , Wj (r) , ei) - f (r, <j> r (Q,Uj(T) ) e i )) + 

+ d i ( r ) (/ ( T : 0r (0 , u 3 - (t) , e<) - / (t, r (£) , u(t) , e,) ) dr 



(40) 



Applying the triangle inequality, Assumption [T] Condition [T] in Corollary [TJ and the boundedness of d, we 
have that there exists a C > such that 
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\\<t>m-U0\\2< / E C |[ rf i]^ r )-^( T )l +i ll^fe)-^(e)|| 2 + i|K-(r)-M(T)|| 2 dr. (41) 



i=i 



Applying the Bcllman-Gronwall Inequality (Lemma 5.6.4 in [29), we have that 

ll<MG) - Mt)h < eL (J C NiW - d ( T )Wi + L ~ u(r)\\ 2 dT) . (42) 

Note that ||u||2 < |M|i for each u G K m . Then applying Holder's inequality (Proposition 6.2 in [13]) to the 
vector valued function, we have: 

\\d 3 (T) - d{r)\\ l dT < \\dj-d\\ L2 , and / \\u 3 (t) - u{r)\\ l dT <\\u 3 - u\\ L2 . (43) 
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Since the sequence 6 converges to 6 for every e > we know there exists some jo such that for all j 
greater than j , ||6 — < e. Therefore ||</>t(6) — 0t(Olla — e L (£ + C)e, which proves the convergence of 
{<M£i)}?==i to ^ f (£) f° r eacn * e [0> 1] as J — * 00 ■ Since this bound does not depend on t, we in fact have 
the uniform convergence of {x^ j ^}°° =1 to as j —> oo, hence obtaining our desired result. □ 

Notice that since X r is a metric space, the previous result proves that the function <f> t which assigns 
£ G X r to <^>t(£) as the solution of Differential Equation (13 1 employing the notation defined in Equation (14) 
is continuous. 



Corollary 2. The function <f> t that maps £ G X r to </>*(£) as the solution of Differential Equation (13) where 
we employ the notation defined in Equation ( 14 ) is continuous for all t G [0, 1] . 

In fact, our arguments have shown that this mapping is Lipschitz continuous: 

Corollary 3. There exists a constant L > such that for each 6)6 G X r and t G [0, 1]: 

11^(6) -<M6)I| 2 <£||6-6IU, (44) 



where 4>t(£) is as defined in Equation (14). 

As a result of this corollary, we immediately have the following results: 

Corollary 4. There exists a constant L > such that for each 6 = (u\,di) G X r , 6 — (^21^2) G <%V> a?l ^ 
i G [0,1]: 



(1) ||/(t,&(fi),ui(*)>di(*)) - /(t,&(£a),ua(t),da(*))| 



< 



< - 611* + - «2(*)lla + MrfxC*) - da(*)ll 2 ). 



(2) 



(3) 



dx 



df, 



t,0t(fl),Ul(t),di(t)) - ^(t,0t(6),U2(t),d 2 (t)) 



< 



< £(||6 - + IM*) - ua(*)|| 2 + - d 2 (t)\\ 2 ), 



df df 

^(t,0 t (6),«i(*),di(t)) - ^(t,M^)Mt),d 2 (t)) 



< 



i,2 



< £(116 - 611* + IM*) - «2(t)|| 2 + \\di(t) - d 2 (t)\\ 2 ), 



where <f>t(£) is as defined in Equation (14). 



Proof. The proof of Condition [T] follows by the fact that the vector field / is Lipschitz in all its arguments, 
as shown in the proof of Theorem [2j and applying Corollary [3j The remaining conditions follow in a similar 
fashion. □ 

Corollary 5. There exists a constant L > such that for each 6)6 G X r , j G J , and t G [0, 1]: 

(1) M<M6))-M<M6))| <£ 116-611*, 

(2) ||^(0i(6))-l l (^(6))|| 2 <£||6-6ll*, 

(3) |/i;(&(6))-/>;(<M6))| < £116-611*, 

(4) ||^W6))-^(^(6))|| 2 < £116-611*, 

where 4>t{C) is as defined in Equation ( |14[ ). 

Proof. This result follows by Assumption [2] and Corollary [3j □ 
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Even though it is a straightforward consequence of Condition [T] in Corollary [5j we write the following 
result to stress its importance. 

Corollary 6. There exists a constant L > such that, for each £1,^2 S %r : 

|J(£iW(6)|<£|Ki-6Lf (45) 



where J is as defined in Equation (15). 

In fact, the ^ is also Lipschitz continuous: 
Lemma 3. There exists a constant L > such that, for each ^1,^2 € Af r : 

|*(&)-*(6)|<£||&-&||* (46) 

where if? is as defined in Equation (|16|. 



Proof. Since the maximum in W is taken over J x [0, 1], which is compact, and the maps (j, t) <— > ipj,t(£) are 
continuous for each £ e A", we know from Condition [3] in Corollary [5] that there exists i > such that, 

-*(&)= max ^i, t (fi)- max Vj.t(6) 

-r.T,,,^^ 1 )"^^ 5 (47) 
(ji)ejx[o,i] 

<£||fr-6ll*. 

By reversing £1 and £2, and applying the same argument we get the desired result. □ 
4.2 Derivation of Algorithm Terms 

Next, we formally derive the components of the optimality function and prove the well-posedness of the 
optimality function. We begin by deriving the formal expression for the directional derivative of the trajectory 
of the switched system. 



Lemma 4. Let £ = (u,d) € X r , £' = (u',d') G X, and let <f> t : X r — > W 1 be as defined in Equation (14). 
Then the directional derivative of 4>t, as defined in Equation (25), is given by 

D<M£; O = J Q <f> [i \t, r) ^ (r, MOMr), d{r))u'(T) + £ /(r, u(r), ei )<(r)j dr, (48) 

where $^(t, t) is £/ie unique solution of the following matrix differential equation: 

^(t,T) = ^(t,MO,u(t),d(t))<Z>(t,r), te[0,l], $(r,r) = /. (49) 
Proof. For notational convenience, let :r/ A ) = a;^ +A ^ ), i/ A ) = w + Ait', and d' A ) = d + Ad'. Then, if we define 



(50) 



Ax^(t) = / f(r, iW(r),«W(r) 1 ^( r )) - /(r, ^(t), t»(r), d(r))dr, 
Jo 

thus, 

Aa;W(t) = f f(T,xW(T),u w (T),dW(T)) - /(r, (t), ii( A ) (r), d(r))dr+ 

+ f f(T,x^(r),u^(T),d(r)) -f(r,x^(t)M X) (r) ) d(T))dr+ 
Jo 

+ f f(T,x^(T)y x \r),d(r)) - f(T,x^(t),u(r),d(T))dT, (51) 
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and applying the Mean Value Theorem, 



Ax^(t) = / A^<(r)/(r ;a; ( A )(T), U ( A )(T), e4 ) + 
Jo ,_, 



+ / if ( r ' x(?) ( T ) + ^WAa; (A) (T), u (A) (r), Ai' A >(r)+ 
Jo " x 



+ ^A^(r, a; «)(r), U (T) + ^(T)A U '(r) ; d(T)) U '(r)dt, (52) 



where : [0, t] —> [0, 1]. 

Let z(t) be the unique solution of the following differential equation: 

z(r) = g (r, a:(«(r), u(r), d(r))z(r) + ^ (r, (r), «(r), d(r))«'(r)+ 



<9u 



+ E<( r )/( r ' x(?) ( T )' u ( r )' e ; re[0,f], z(0)=0. (53) 



Aa' A '(t) 



We want to show that lim^o 
follow from Condition [2] in Assumption [l] 



i=l 



^(t) = 0. To prove this, consider the following inequalities that 
I2 



< 



< 



< L 



dx 



z(r) 



Ax^(t) 



dr+ ( L(\\iy x (T)Ax^(T)\\ 2 + X\\u'(T)\\^\\z(t)\\ 2 dT 



z(t)- 



Ax^(t) 



2 -jo 

dr + L f (\\AxW(T)\\ 2 +X\\u'(T)\\ 2 )\\z(t)\\ 2 dT, 
2 Jo v " 7 



also from Condition [3] in Assumption [Tj 

f {%(^ xii Hr),u(T),d(r)) - ^-( T ,x^(r),u(r) + ^(r)A U '(r), d(r))^) u'(r) 



(54) 



< 



<L \\\v u (T)u'(T)\\ 2 \\u'{T)\\ 2 dT < L / A||t/(T)||adT-, (55) 



2 ii" v ;ii2 / 11 'H2 ' 

•/ 



and from Condition [T] in Assumption [T] 



£ <-(r) (/(r, x«)(r), u(r), e z ) - /(r, ^'(r), u^(r), 
i=i 

<L /■ t ^d;(r)(||A a ;W(r)|| 2 + A|| U '(r)|| 2 )dT. (56) 

J° i=l 

Now, using the Bellman-Gronwall Inequality (Lemma 5.6.4 in [29 ) and the inequalities above, 
{t) - M < e U L[ f (|| Ax( A )(r)|| 2 + A \\u'(r)\\ 2 ) \\z(t)\\ 2 + A \\u'(r)\\l + 

+ ^^(r)(||A a; ( A )(r)|| 2 + A|| M '(r)|| 2 )dr), (57) 



A 



-z(t) 
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but note that every term in the integral above is bounded, and Ax^lr) — > for each r € [0,t] since 
a?M — > a;® uniformly as shown in Lemma [5J thus by the Dominated Convergence Theorem (Theorem 2.24 
in [13] ) and by noting that D</> t (£;£'), as defined in Equation (48), is exactly the solution of Differential 
Equation (53) we get: 



lim 



Ax^(t) 



z{t) 



lim 

A^O A 



h\x^ +x ^[t) - x^[t) - D(f> t (£-,\£')\\ = 0. 



The result of the Lemma then follows. 

Next, we prove that D</> t is bounded by proving that $w is bounded: 
Corollary 7. There exists a constant C > suc/i i/iai /or eac/i t,T E [0, 1] and £ 6 A",, 

||$ (e) (t,r)||. !2 <C, 

where r) is £/ie solution to Differential Equation (49). 

Proof. Notice that, since the induced matrix norm is submultiplicative, 



(58) 



□ 



(59) 



^\t,T) 



i,2 



<9x 



,x ( « ) (s),w(s),d(s))$ (4) (s,r) ds 



< 1 + 



' df 



dx 



(s,x^(s),u(s),d(s)) 



i.2 



(60) 



i,2 



where in the last step we employed Condition [T] from Corollary [T] with a constant C > and the Bellman- 
Gronwall Inequality. □ 



Corollary 8. There exists a constant C > such that for all £ G AV, £' G X , and t G [0, 1]: 

||D«M£;£')|| 2 <Cni*, 



(61) 



where ~D(f>t is as defined in Equation (48). 

Proof. This result follows by employing the Cauchy-Schwarz Inequality, Corollary [T] and Corollary [7] □ 

In fact, we can actually prove the Lipschitz continuity of 
Lemma 5. There exists a constant L > swc/i that for each £i,£ 2 € A" r and eac/i t,T £ [0, 1]: 

*«*>( t>T )_$to)( t)T ) ^ <i||ei-6IU, (62) 



where $w is the solution to Differential Equation (49). 

Proof. Letting £1 = (tti,di) G A r and £2 = ("2,^2) € A r and by applying the Triangular Inequality and 
noticing the induced matrix norm is compatible, observe: 



$(« 1 )(i,T)-$« 2 )(t,T) 



< 



df 

dx 



df 

d. 



s, a; (?2) (s),u 2 (s),d 2 (s) 



j - [s,x^(s),u 1 (s),d 1 (s)) - g (s, z«»>(a), ua^.daCs) 



(.2 



i> (?l) (s,r) - $ ( « 2) (s,r) 



ds+ 



$ (6) (s,r) ds. (63) 



i,2 



By applying Condition [T] in Corollary [T] Condition [2] in Corollary |4j Corollary [7J the same argument as in 
Equation (43), and the Bellman- Gronwall Inequality (Lemma 5.6.4 in [H]), our desired result follows. □ 
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A simple extension of our previous argument shows that for all t £ [0, 1], D0 t (£; •) is Lipschitz continuous 
with respect to its point of evaluation, £. 

Lemma 6. There exists a constant L > such that for each fij £2 G %r> C G % > an d t G [0j 1] ■' 

l|D0 t (6;r)-D^(6;OII 2 <^liei-6IUIinU (64) 

where D(j>t is as defined in Equation 1481). 



Proof. Let fi = (ui,di), £2 = (""2,^2), and £' = (u',df). Then, by applying the Triangular Inequality, and 
noticing that the induced matrix norm is compatible, observe: 



1.2 



Of 



du 



(s,:r^)(s),ui(s),di(s)) 



+ 



|K(s)|| 2 ds+ 



1,2 



t 9 



+ 



||D^(a ; e')-D^(6;OII 2 < / I «i> ,;:l c..s; - <i.'^'i/..,i 

$ (&) (i,s) 

+ J Q E( |* Cei) (t s a)-* (fe) (*.*) i2 /(s,^)( S ), Ul ( S ), ei ) 

+ |$« a >(t,«) /( fl ,a;« 1 )(s),ui(«),c i )-/(«,x« a )(«),«a(s) ) e i ) ] ||<f(s)|| da. (65) 

II *,2 2 / 

By applying Corollary [7J Condition [T] in Corollary [T] Lemma [5j Conditions [T] and [3] in Corollary |4j together 
with the boundedness of u'(s) and d'(s), and an argument identical to the one used in Equation ( |43[ ), our 
desired result follows. □ 

Next, we prove that D0 t is simultaneously continuous with respect to both of its arguments. 

Lemma 7. For each t £ [0,1], £ £ AV, and £' £ X, the map (£,£') M> D</> t (£;£'), as defined in Equation 
(48), is continuous. 

Proof. To prove this result, we can employ an argument identical to the one used in the proof of Lemma [2] 
First, note that u(t) £ U for each t £ [0, 1]. Second, note that /, and ^ are bounded, as shown 

in Corollary [7] and Condition [l] in Corollary lj Third, recall that /, and ^ are Lipschitz continuous, 
as proven in Lemma [5] and Conditions [T] and^S] in Corolla ry |4| respectively. Finally, the result follows after 
using an argument identical to the one used in Equation ( |43| . □ 

We can now construct the directional derivative of the cost J and prove it is Lipschitz continuous. 



Lemma 8. Let £ £ X r , £' £ X, and J be as defined in Equation (15). Then the directional derivative of the 
cost J in the £' direction is: 

DJ(£;O = ^(<M0)D<M£;O- (66) 
Proof. The result follows directly by the Chain Rule and Lemma [4j □ 
Corollary 9. There exists a constant L > such that for each £1,^2 £ X r and £' £ X: 

|DJ(&;0 - DJ($,;OI < £ ||6 - U']\ x , (67) 



where DJ is as defined in Equation (66). 
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Proof. Notice by the Triangular Inequality and the Cauchy-Schwartz Inequality: 

dh , 



|Dj(^;0-DJ(6;e')l < 



dx 



(Mil)) 



\DMti;?)-vMte?)\\ 



Ox 



|D0i(6;Oll 2 - (68) 



The result then follows by applying Condition [2] in Corollary [T] Condition [2] in Corollary [5j Corollary [8j and 
Lemma [6l □ 

Next, we prove that D J is simultaneously continuous with respect to both of its arguments, which is a 
direct consequence of Lemma [7| 



Corollary 10. For each £ G X r and £' G A", </ie map (£,£') i-> DJ(£;£'), as defined in Equation (66), is 
contmMOMS. 

Next, we construct the directional derivative of each of the component constraint functions ipjj and prove 
that each of the component constraints is Lipschitz continuous. 



Lemma 9. Let £ G X r , £' G X , and ipj t defined as in Equation (17). Then for each j G J and t G [0, 1], 
the directional derivative of ipjj , denoted Dipj,t , is given by: 



Bh ■ 

d <M£;£0 = ^(<M£))d<M£;£')- 



(69) 

Proof. The result follows using the Chain Rule and Lemma |4j □ 
Corollary 11. There exists a constant L > such that for each £1,^2 G X r , £' G X , and t G [0, 1]: 

|D^- t (a;n-D^(6;Ol <£||6-6IL*ril*, (70) 



where Dipj.t is as defined in Equation (69). 

Proof. Notice by the Triangular Inequality and the Cauchy Schwartz Inequality: 

dhj 



|D<M?i;0-D?M6;r)l < 



dx 



(Mb)) 



|D<M6;0-D<M6;OII, 



^(*fo))-^(*(60) 



P0t(&;Olla- ( 71 ) 



The result then follows by applying Condition [3] in Corollary [T] Condition [2] in Corollary [5j Corollary [8j and 
Lemma [6J □ 

Next, we prove that Dipj t is simultaneously continuous with respect to both of its arguments, which 
follows directly from Lemma [7] 

Corollary 12. For each £ G X r , £' G A", andt G [0, 1], tte map (£,£') i-> D^/>j )t (£; £'), as defined in Equation 
(69), is continuous. 

Given these results, we can begin describing the properties satisfied by the optimality function: 



Lemma 10. Let C, be defined as in Equation |31J). Then there exists a constant L > such that, for each 
£i>£a>£ G AV, 

IC(£i,0-C(6,OI<£||6-6ll*- (72) 
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Proof. To prove the result, first notice that for {xi}i^x, {yi}i<£x C 



max x, 



< max \xi\ , and maxij — maxy^ < max{xi — yA. 



Therefore, 



max Xi — max yi 



< max \Xi - Vi 



Letting = max{0, *(£)} and = max{0, -#(£)}, observe: 

C(£, O = max (DJ(£; *+(£), max D^,t«;C - "7*"(0 
I jej, te[o,i] 



.V- 



(73) 
(74) 

(75) 



Employing Equation ( 74 ) 



|C(&, O - C(6, O | < maxj |D J(&; £' - 6) " D J(&; ^ - 6) | + - 



(&)|, 



max |D^, t (a;C' - 6) - D^, t (6;C - 6) + 7 - + \W - Si\\x - IIC - 6IU • 

(76) 

We show three results that taken together with the Triangular Inequality prove the desired result. First, 
by applying the reverse triangle inequality: 



Second, 



\Z'-ti\\x-M'-b\\x\<Mi-b\\ X . (77) 



|DJ(6; - Ci) - D Jfe; £' - 6)| = |D J(6; - 6) - D J(&; £' - &) + DJ(&; & - &)| 

< | D J(& ; £' ) - D J(6 ; O | + | DJfo ; 6 ) - D Jfe ; & ) | + 

|f(fc(6))Dfc(&;6-fc)| (78) 

<£ 116-611*, 

where i > and we employed the linearity of D J, Corollary [9j the fact that £' and 6 are bounded since 
^ the Cauchy-Schwartz Inequality, Condition ffl in Corollary [TJ and Corollary [8| Notice that by 
employing an argument identical to Equation ( 78 ) and Corollary |11| we can assume without loss of generality 
that |D^, t (a;£'-&)-Dfe(£2;£'-£2)| <L$i-£ 2 \\ x - Finally, notice that by applying Lemma § *+(£) 
and are Lipschitz continuous. □ 

In fact, £ satisfies an even more important property: 



Lemma 11. For each £ 6 X p , the map £' t— > C(£>£')> as defined in Equation (31 1, is strictly convex. 



Proof The proof follows after noting that the maps £' i-> DJ(£; C - £) and £' h-> ^ (<MO) D( M£; f - are 
affine, hence any maximum among these function is convex, and the map £' i— > — is strictly convex 
since we chose the 2-norm as our finite dimensional norm. □ 

The following theorem, which follows as a result of the previous lemma, is fundamental to our result 



since it shows that g, as defined in Equation (301, is a well-defined function. We omit the proof since it 



is a particular case of a well known result regarding the existence of unique minimizcrs of strictly convex 
functions over bounded sets in Hilbcrt spaces (Proposition II. 1.2 in llj). 



Theorem 4. For each £ G X p , the map £' i— > C(£>£')> as defined in Equation (31), has a unique minimizer. 
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Employing these results we can prove the continuity of the optimality function. This result is not strictly 
required in order to prove the convergence of Algorithm [TJ or in order to prove that the optimality function 
encodes local minimizcrs of the Switched System Optimal Control Problem, but is useful when we describe 
the implementation of our algorithm. 



Lemma 12. The function 9, as defined in Equation (30), is continuous. 

Proof. First, we show that 9 is upper semi-continuous. Consider a sequence C X r converging to 

£, and £' G X r such that = C(£>£')> i- e - C — where g is defined as in Equation (30 1. Since 

^)<Cte,e')for a UiGN, 

limsup0(&) < IimsupC(6,0 = C&O = 0(0, (79) 

which proves the upper semi-continuity of 9. 

Second, we show that 9 is lower semi-continuous. Let {£i}»£N such that = C(&i£i)> i.e. £| = 

g(£i). From Lemma 10 we know there exists a Lipschitz constant L > such that for each i € N, 
~ Cte,e-)l < - Consequently, 

0(0 < (ac, & - c(6. eo) + cte, $ < xii* - &iu + ( 8 °) 

Taking limits we conclude that 

0(0<]iminf0(&), (81) 
which proves the lower semi-continuity of 9, and our desired result. □ 
Finally, we can prove that 9 encodes a necessary condition for optimality: 



Theorem 5. Let 9 be as defined in Equation (30), then: 

(1) 9 is non-positive valued, and 

(2) If {; £ X p is a local minimizer of the Switched System Optimal Control Problem as in Definition^ then 

0(O-o. 

Proof. Notice that = 0, therefore 0(£) = min ee;t v C(£,£') < ((£,€) = 0- This proves Condition [TJ 

To prove Condition [2] we begin by making several observations. Given £' € X r and A € [0, 1], using the 
Mean Value Theorem and Corollary [9] we have that there exists s <E (0, 1) and L > such that 

+ A(£' - 0) - J(0 = D J(£ + ,sA(C' - 0; A(0 - 0) 

<ADj(e ; e'-e) + iA 2 ne'-eii^. 

Letting A(£) — {(j,t) £ J x [0, 1] | = ftj(a;^(t))}, similar to the equation above, there exists a pair 
(j,t) € .A(f + A(£' - 0) and s e (0, 1) such that, using Corollary [Tlj 

*(e + A(r - o) - *(o < + a(o - o) - *(o 
<^,t(e+A^'-o)-^,t(e) 
= D^, t (e+sA(e'-o;A(e'-e)) 

<AD^, t (£;0-0+ L A 2 ||£'-£ll^ 
Finally, letting L denote the Lipschitz constant as in Condition [TJ in Assumption [2] notice: 
*(£ + A(£'-0) -*(£)= max , + KC - 0) - max ^,t(0 

" a, t )^[o,i] fe (C + A(e ' ~ C)) " ( 84 ) 
<imax ||^(e + A(C-O)-0t(O|| 2 . 



(83) 
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We prove Condition [5] by contradiction. That is, using Definition [3j we assume that 0(0 < and show 
that for each e > there exists | <G Af w (£,e) (1 {{; £ X p \ < 0} such that J(£) < J(0, where Af w (£,e) is 
as defined in Equation ( |29[ ), hence arriving at a contradiction. 

Before arriving at this contradiction, we make three initial observations. First, notice that since (£/f p 
is a local minimizer of the Switched System Optimal Control Problem, ^(0 < 0. Second, consider g as 
defined in Equation (30), which exists by Theorem[4j and notice that since < 0, <?(£) ^ £. Third, notice 
that, as a result of Theorem[3j for each (£ + A(</(0 — 0) € and e' > there exists a ^ e ^ such that 

I^CCx) _ a; «+AC ff «)-0)|| ioo <e > (85) 

where z«) is the solution to Differential Equation fL3|. 
Now, letting e' = — > and using Corollaryjsj 

|| T (6J _ r (l)|| < || t («a) _ T (S+A(g(|)-0)|| i || T (e+A(3(€)-0) _ -(Oil 

Next, observe that: 

6(0 = max |d J(C; - 0. ( . t Q x] D ^,t(C; <?(0 - + 7* (0} + 11.9(0 - ClU < 0- (87) 
Also, by Equations p2| ), ( |85| , and | |87[ ), together with Condition [I] in Assumption [2] and Corollary [3] 

J(60 - AO < A60 - ■/(£ + Hg(0 - 0) + J (e + - 0) - AO 

< l ii^ (e A ) -M€ + x (g(0 - 0)ll 2 + + 4 ^a 2 

< J Le' + 6 l (£)A + 4A 2 LA 2 ( 88 ) 

< ^+4^^ 

where A = max{||u|| 2 + 1 | u e U} and we used the fact that ||£ - < 4A 2 and DJ(£;£' - £) < 0(£). 
Hence for each A e U), , 

J(60 - J(0 < o. (89) 

Similarly, using Condition [TJ in Assumption [2j together with Equations (83), (84), and (87), we have: 

*(60 < - *(€ + A(s(0 - 0) + *(£ + A(g(0 - 0) 

< L max ||<M6,) - MS + A(fl(0 - 0)ll 2 + * (0 + (*(0 - 7*(0)A + AA 2 L\ 2 

t€[0,l] 

< Le' + 6>(0A + 4A 2 LA 2 + (1 - 7 A)*(£) ' !>0 ' 

< ^ +4 A 2 LA 2 + (l- 7 A)*(e), 

where A = max {||u|| 2 + 1 | u G U) and we used the fact that ||£ - £\\ 2 X < AA 2 and D^ )t (£;f' - < 
6>(0 -7*(0 for each ejx [0,1]. Hence for each A e fo,min |}): 

< (1 - 7A)*(0 < 0. (91) 
Summarizing, suppose £ € X p is a local minimizer of the Switched System Optimal Control Problem and 
0(0 < 0. For each e > 0, by choosing any 

K°H^? 2xW-L-*«) })' <92) 



we can construct a £a € such that £a € A/"u,(£,e), by Equation ( |86| ), such that J(£a) < AO; by Equation 
(89), and ^(£a) < 0, by Equation (91 ). Therefore, £ is not a local minimizer of the Switched System Optimal 
Control Problem, which is a contradiction and proves Condition [2] □ 
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4.3 Approximating Relaxed Inputs 

In this subsection, we prove that the projection operation, pjv, allows us to control the quality of approxi- 
mation between the trajectories generated by a relaxed discrete input and its projection. First, we prove for 
d G XV, ^jv(d) e V r and V N (F N {d)) G V p : 



Lemma 13. Let d G X> r , J^at fee as defined in Equation (34), and 'P/v oe as defined in Equation (35). TTien 
/or eac/i iV G N and t G [0, 1] : 

(1) [Jjv(rf)]i(i) € [0,1], 

(2) EU-Mrf)k(t) = 1, 

(3) [^(^(^.(^{o.i}, 

(4) Ei=i [^(^r(d))] i (*) = l- 

Proof. Condition [l] follows due to the result in Section 3.3 in [TH]. Condition [2] follows since the wavelet 
approximation is linear, thus, 

N 2 fc -l , 

Oki 



7 (93) 

AT 2 fc ~l 



llfc. .112 



fc=0 j=0 



fcill L 2 



where the last equality holds since =0 for each 

Conditions [3] and [4] are direct consequences of the definition of 'P/v, since Pat can only take the values 
or 1, and only one coordinate is equal to 1 at any given time t G [0, 1]. □ 

Recall that in order to avoid the introduction of additional notation, we let the coordinate- wise application 
of J-jv to some relaxed discrete input d G T> r be denoted as J-jv(cZ) and similarly for some continuous input 
ueU, but in fact F N as originally defined took L 2 ([0, 1],E) D BV([0, 1],R) to i 2 ([0, 1],M) D BV([0, 1],R). 
Next, we prove that the wavelet approximation allows us to control the quality of approximation: 

Lemma 14. Let f G L 2 ([0, 1], R) D BV([0, 1], R), then 

\\f-?*(J)h><l(^) N \\f\\BV, (94) 



where Tm is as defined in Equation ( 34 ) . 

Proof. Since L? is a Hilbert space and the collection {bkj}k,j is a basis, then 



oo 2"-! 



fe=0 j=0 



Note that ||6 fej || 2 2 = 2~ fc and that 



ifte [j2- fc ,( J + i)2- fc ), 

= /. M«)^ = ^ -* + U + l)2- fc if t G [(j + |) 2- fe , (j + 1) 2- fe ) , (96) 

otherwise, 
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thus ||ufcj||z,o° = 2 k . Now, using integration by parts, and since / G BV([0, 1],I 



I </,*«>! = 



0" + l)2" 



J2- 



f(t)v kj {t)dt 



< 2 



-fc-i 



(3+1)2" 



I /(*)!* 



(97) 



J'2- 



Finally, Parseval's Identity for Hilbert spaces (Theorem 5.27 in [T3]) implies that 



oo 2 K -1 



K/,MI S 



L 2 X] X! II/,, . 2 

oo 2 fc -l 

< E 2 " fc " 2 E 

fc=Af+l j=0 



+ 1)2" 



\f(t)\dt 



J2- 



< 2 



— JV— 2 II JM|2 



\BV> 



as desired. 



(98) 



□ 



The following lemma is fundamental to find a rate of convergence for the approximation of the solution 
of differential equations using relaxed inputs: 



Lemma 15. There exists K > such that for each d G V r and f e L 2 ([0, 1],W) D BV([0, 1],K 9 ), 

N /-^ N 

2 



\(d-v N (F N (d))j)\<KU-^y \\f\\ L2 \\d\\ BV +(i 



ii/n 



BY 



(99) 



where is as defined Equation (34 1 and Vn is as defined in Equation ( |35| ). 

Proof. To simplify our notation, let t k = ^r, p ik = [J 7 jv(d)]j(tfe), = X^=iPjfc> and 



.4 



ik 



Also let us denote the indicator function of the set Ai k by tA ik ■ Consider 

2™-l 1 H k 

/ 

>t k 

Let Wik '■ [0, 1] — > R be defined by 



(j-/v(rf)-p/v(^(rf)),/) = E E / (pi*-w*)) /<(*)* 



(100) 



(101) 



Wife 



(t) = / Pife - t Aik (s)ds = < ^rPikS{i-i) k + (Pifc - 1) (t - tfc - 2*r.%-i)fc) if * G Afc, 

[ ^wPik{Sik — 1) + Pife (* — ifc ~ 2^" Sik) if * € [tfe + ^rSik,tk+i) , 

(102) 

when t G [tfe, tfe+i], and Wi k (t) = otherwise. Note that ||tWifc||i,oo < pgr. Thus, using integration by parts, 



tk+i 



(p ik -l Aih (t))fi(t)dt 



tfe + 1 



w{t)f\{t)dt 



< 



2 w 



tfc + 1 



I /<(*)!*, 



and 



i 2 -i r tk+i i 

\(T N (d)-V N (T N (d))j)\ < ^ E / E^l^wi* 

fc=0 tk i=l 



(103) 



(104) 



< pHI/llw- 
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where the last inequality follows by Holder's Inequality. 



Also, by Lemma 14 we have that 



-(- 
2VV2 



JV 



H\\BV- 



Hence, using Cauchy- Schwartz's Inequality, 

\(d-V N {T N (d)),f)\ < \\d-T N (d)\\ L 2\\f\\ L 2 + \(T N (d)-V N {T N (d))j)\, 



and the desired result follows from Equations (|98|, (|104| 
Note that Lemma 



(105) 

(106) 

□ 
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does not prove convergence of T^n^n^)) to d in the weak topology on T> r . Such 
a result is indeed true, i.e. 7 7 /v(-?iv(c0) does converge in the weak topology to d, and it can be shown using 
an argument similar to the one used in Lemma 1 in [36J. The reason we chose to prove a weaker result is 
because in this case we get an explicit rate of convergence, which is fundamental to the construction of our 
optimization algorithm because it allows us to bound the quality of approximation of the state trajectory. 



Theorem 6. Let be defined as in Equation (36) and tp t be defined as in Equation (14). Then there exists 
K > such that for each £ = (u, d) € X r and for each t £ [0, 1], 



\Mpn(0)~M0\\ 2 <k 



N 



{U\\bv + 1). 



(107) 



Proof. To simplify our notation, let us denote un — J-jv(w) and d^ = / P_/v(.FAr(<i)) , thus Pn(£) = (un, dzv)- 
Consider 



\\x iuN - dN) {t) - x M (t)\\ 2 < \\x {uN ' dN) (t) - x {u - dN \t)\\ 2 + \\x {u - dN \t) - x [u - d) (t)\\ r 

The main result of the theorem will follow from upper bounds from each of these two parts. 
Note that 



(108) 



x (u N ,d N )^ _ a;(M*)(t)|| < / ||/( Sj x^' dN Hs),u N (s),d N (s)) - f(s, x^ d -\s), u(s), d N (s)) \\ 2 ds 



< L 



(u N .d N ) , 



(s) - x^\s)\\ 9 + \\u N {s) - u{s)\lds, 



(109) 

thus, using Bellman-GronwalPs Inequality (Lemma 5.6.4 in |29| ) together with the result in Lemma |14| we 
get 



S UN > dN \t) - x( u ' dN \t)\L < 



Le L V2 ( 1 



V2 



N 



\u\\bv 



(110) 



On the other hand, 



t i 



^2([d N ]i(s) - di(s))f(s, x^is), u(s), e t )ds+ 
q 

X>Ar]i(«) (/( s > x(U ' dN) ( s )' u ( s )' e " /( s ' ^ (u ' d) (s), «(*), e,)) ds, (111) 



thus, 



\x 



(u,d N ) 1 



1 Q 



^2([d N ]i(s) - *(!))/(«,i M W,tiW 1 ei)i« 



i=i 



+ 



+ L / \\x iu ' dN) {s)-x iu - d) (s)\lds. (112) 
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Using Bellman-Gronwall's inequality we get 



\x 



(u,d N ) I 



(t) - x^ d \t)\\< e 1 



1 9 



]T ([d N ]i(s) - <k{s))f(t, a?M (s),u(s),ei)ds 



(113) 



Recall that / maps to E™, so let us denote the k-ih coordinate of / by ft- Let Vki(t) = fk (t, x^ u ' d '(t), u(t), e^) 
and Vk — (wfcii ■ • ■ , Vk q ), then is of bounded variation. Indeed, by Theorem [I] and Condition [I] in Corollary 
[11 we have that ||a;^|| B y < C. Thus, by Condition [l] in Assumption [l] and again using Theorem [TJ we get 
that, for each i £ Q, 

\\v kl \\ BV <L(l + C+\\u\\ BV ). (114) 

Moreover, Condition [l] in Corollary [I] directly imply that ||«fci||i, 2 < C. Hence, Lemma 15 implies that there 
exists K > such that 



\(d-d N ,v k )\<K((^j C\\d\\ BV + q (^ 



(i + c + Hw) 



(115) 



Since Equation (115) is satisfied for each k £ {1, . . . , n}, then after ordering the constants and noting that 
2 N > 2^ for each N £ N, together with Equation (110) we get the desired result. 



□ 



4.4 Convergence of the Algorithm 

To prove the convergence of our algorithm, we employ a technique similar to the one prescribed in Section 
1.2 in 29 . Summarizing the technique, one can think of an algorithm as discrete-time dynamical system, 
whose desired stable equilibria are characterized by the stationary points of its optimality function, i.e. 
points £ £ X p where #(£) = 0, since we know from Theorem [5] that all local minimizers are stationary. Before 
applying this line of reasoning to our algorithm, we present a simplified version of this argument for a general 
unconstrained optimization problem. This is done in the interest of clarity. Inspired by the stability analysis 
of dynamical systems, a sufficient condition for the convergence of our algorithm can be formulated by 
requiring that the cost function satisfy a notion of sufficient descent with respect to an optimality function: 

Definition 4. Let S be a metric space, and consider the problem of minimizing the cost function J : S — > 1R. 
We say that a function T : S —¥ S has the sufficient descent property with respect to an optimality function 
(— oo,0] if for each x £ S with 9(x) < 0, there exists a S x > and O x C S , a neighborhood of x, 
such that: 

J(T(x')) - J{x') < -S x , W £ O x . (116) 

Importantly, a function satisfying the sufficient property can be proven to approach the zeros of the 
optimality function: 

Theorem 7 (Theorem 1.2.8 in |29j). Consider the problem of minimizing a cost function J : S — >• K. 
Suppose that S is a a metric space and a function T : S —> S has the sufficient descent property with respect 
to an optimality function 8 : S — > (— oo, 0], as described in Definition^ Let {xjjjgpj be a sequence such that, 
for each j £ N: 

\r(x 3 ) ife( Xj )<o, 



Cj+l = 



if6(xj 



0. 



(117) 



Then every accumulation point of {xj}j^jq belongs to the set of zeros of the optimality function 9. 

Theorem[7j as originally stated in [29) . requires S to be a Euclidean space, but the result as presented here 
can be proven without requiring this property using the same original argument. Though Theorem [7] proves 
that the accumulation point of a sequence generated by T converges to a stationary point of the optimality 
function, it does not prove the existence of the accumulation point. This is in general not a problem for 
finite-dimensional optimization problems since the level sets of the cost function are usually compact, thus 
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every sequence produced by a descent method has at least one accumulation point. On the other hand, 
infinite-dimensional problems, such as optimal control problems, do not have this property, since bounded 
sets may not be compact in infinite-dimensional vector spaces. Thus, even though Theorem [7] can be applied 
to both finite-dimensional and infinite-dimensional optimization problems, the result is much weaker in the 
latter case. 

The issue mentioned above has been addressed several times in the literature [1113011101111], by formulating 
a stronger version of sufficient descent: 

Definition 5 (Definition 2.1 in [5]). Let S be a metric space, and consider the problem of minimizing the 
cost function J : S — > R. A function T : S — > S has the uniform sufficient descent property with respect to 
an optimality function 9 : S — > (— oo,0] if for each C > there exists a 8c > such that, for every i£<S 
with 9(x) < 0, 

J(r(x)) - J(x) < -S c . (118) 

A sequence of points generated by an algorithm satisfying this property, under mild assumptions, can be 
shown to approach the zeros of the optimality function: 

Theorem 8 (Proposition 2.1 in Consider the problem of minimizing a lower bounded cost function 

J : S — > [a,oo). Suppose that S is a a metric space and T : S — > S satisfies the uniform sufficient descent 
property with respect to an optimality function 9 : S — > (— oo,0], as stated in Definition^ Let {xj}j<^m be a 
sequence such that, for each j £ N: 

frfe) «»(,)<», 

I Xj if 0(Xj) = (J. 



Then, 



lim 9(xj) = 0. (120) 



Proof. Suppose that liminfj_ > . 00 9(xj) — — 2e < 0. Then there exists a subsequence {^j fc }jteN such that 
9(xj k ) < —e for each k g N. Definition [5] implies that there exists S e such that 

J(x jk+1 ) - J(x jk ) < -S e , V/ceN. (121) 

But this is a contradiction, since J(xj+\) < J(xj) for each j £ N, thus J(x.j) — > — oo as j — > oo, contrary to 
the assumption that J is lower bounded. □ 

Note that Theorem [8] does not assume the existence of accumulation points of the sequence {xjjjgN- 
Thus, this Theorem remains valid even when the sequence generated by T does not have accumulation 
points. This becomes tremendously useful in infinite-dimensional problems where the level sets of the cost 
function may not be compact. Though we include these results for the sake of completeness of presentation, 
our proof of convergence of the sequence of points generated by Algorithm [l] does not make explicit use of 
Theorem [8j The line of argument is similar, but our approach, as described in Theorem [9] requires special 
treatment due to the projection operation, p?q, as defined in Equation ( |36[ ) and the existence of constraints. 
Now, we begin the convergence proof of Algorithm [T] by showing that the Armijo algorithm, as defined 



in Equation (32 1, terminates after a finite number of steps and its value is bounded. 



Lemma 16. Let a G (0, 1) and /3 € (0, 1). For every 8 > there exists an M| < oo such that if #(£) < —8 



for £ 6 X p , then fi{^) < M$ , where 9 is as defined in Equation (30) and /i is as defined in Equation (132 



Proof. Given £' £ X and A £ [0, 1], using the Mean Value Theorem and Corollary [9] we have that there exists 
s £ (0, 1) such that 

J(£ + A(£' - 0) - J(0 - dj(£ + 8 \(? - 0; HC - 0) 

<\m(z-e-t)+L\ 2 \\e-t\\ 2 x - 
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Letting A(£) = {(j,t) €jx [0,1] | *(£) = hj(x^(t))}, then there exists a pair (j,t) € A(t, + A(£' - £)) 



and s £ (0, 1) such that, using Corollary 11 



<^,t(e+A(r-o)-^,t(e) 

= D^ it (^ + S A(^-0;A(?'-0) 
<AD^, t (£;£'-£)+£A 2 ||£'-£|| 



.v- 



Now let us assume that < 0, and consider g as defined in Equation (30 1. Then 

0(£)= max (dJ(£; <?(£)-£) max DtM6»(0 "0 +7*(o} < 
I 0,t)e^x[o,i] J 



and using Equation ([122]), 

where A = max {||u||2 + l|u€i/}. Hence, for each k € N such that < ^"j^ we have that 

■7(£ + j8 fc GKO-O)-^(O<aj8 fc 0(O- 

Similarly, using Equations ( |123| ) and ( |124[ ), 

hence for each fceN such that f3 k < min j ^"jf , ^ j we have that 

*(£ + P k (g(0 - 0) - a P k0 (O < (! - 0*7) * (0 < 0. 

If > then 

max Dfe(f iS (()-0<e(a<4, 

0,t)Gjx[0,l] 

thus, from Equation ( |123[ ), 

*(C + 0*(s(O ~ 0) - *(f) - "/^(O < -(1 - ")^ fc + 4^ 2 L/3 2fc . 
Hence, for each k <E N such that /3 fe < ^jr^ we have that 



*(e + /? fe (g(0-o)-*(e)<«/3^(?). 



Finally, let 



M* = 1 + max ^ lo 



(l-oQ5 
4A 2 L 



, be 



then from Equations (126), (128), and (131), we get that /x(£) < M| as desired. 



(123) 



(124) 
(125) 

(126) 
(127) 

(128) 
(129) 

(130) 

(131) 

(132) 

□ 



Next, we show that the determination of the frequency at which to perform pulse width modulation as 
defined in Equation ( |37[ ) terminates after a finite number of steps. 

Lemma 17. Let a € (0,1), a € (0,oo), /3 € (0,1), /3 £ (vf. 1 ); and £ e ^p- < 0> ^ e7J K£) < °°, 

where 9 is as defined in Equation ( 30 ) and v is as defined in Equation ( 37 1 . 
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Proof. Throughout the proof, we leave out the natural inclusion taking £G X p to £ € X r . To simplify our 
notation let us denote M = and £' = £ + f3 M '(<?(£) — £). Theorem pi implies that there exists if > 
such that 



■/(MO) - AO < KL (\\£\\ BV + 1), 



(133) 



where L is the constant defined in Assumption [2j 

Let A{£) = {(j,t) G {1,...,N C } x [0, 1] | = hj(ar^(t))}, then for each pair (j,f) G A(p N (?)) we 
have that 

<feM£'))-^-,*(r) 

JV 



1 



(134) 



din 



1 



Recall that a G (0,oo), (3 G M=,lj, 
N>N 

1 



and ui G (0,1), hence there exists JV e N such that, for each 



KL 



(72T (\\?\\Bv + i)<-<*P N m- 



Also, there exists Ni > No such that, for each N > Ni, 
Now suppose that < 0, then, for each N > N\, 

■/(MO) m = J(pn(0) - AO + AO - AO 

<(af3 M -af3 N )6(Z), 

and 



*MO) = *MO)-*(0 + *(0 

< o. 

Similarly, if ^(^ > then, using the same argument as above, we have that 

*(MO) - < (aP M - a[3 N ) 6(0- 

Therefore, from Equations (137), < |138 ), and ( 139| , it follows that i>(£) < Ni as desired. 

The following lemma proves that, once Algorithm[T]fmds a feasible point, every point generated afterwards 
is also feasible. We omit the proof since it follows directly from the definition of v in Equation (37). 



(135) 
(136) 

(137) 
(138) 

(139) 

□ 



Lemma 18. Let T be defined as in Equation (38) and let '5 be as defined in Equation (16). Let {£,i}ieiq be 
a sequence generated by Algorithm^ If there exists io G N such that ^f(Ci ) < 0, then ^(£i) < for each 
i > i . 

Employing these preceding results, we can prove the convergence of Algorithm [l] to a point that satisfies 
our optimality condition by employing an argument similar to the one used in the proof of Theorem [8] 

Theorem 9. Let 9 be defined as in Equation (30). If {S,i}ieN is a sequence generated by Algorithm^ then 
linij^oo = 0. 
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Proof. If the sequence produced by Algorithm [T] is finite, then the theorem is trivially satisfied, so we assume 
that the sequence is infinite. 

Suppose the theorem is not true, then lim infj-^oo = — 2<5 < and therefore there exists to£N and 
a subsequence {£j,.}fceN such that 0(£j fc ) < —S for each k > k . Also, recall that was chosen such that, 
given //(£), 



(140) 



where u € (0, 1) isa parameter 



From Lemma 



1G 



we know that there exists M|, which depends on 5, such that /J^) > /3 M s . Suppose 
that the subsequence is eventually feasible, then, by Lemma 18 without loss of generality we can 



assume that the sequence is always feasible. Thus, given T as defined in Equation (381, 

< -ujaP M >5. 



(141) 



This inequality, together with the fact that J(£i+i) < J(£i) for each i 6 N, implies that liminf^oo J(£i k ) = 
— oo, but this is a contradiction since J is lower bounded, which follows from Condition [T] in Corollary [5] 

The case when the sequence is never feasible is analogous after noting that, since the subsequence is 
infeasible, then &(£i k ) > for each k € N, establishing a similar contradiction. □ 



5 Implementable Algorithm 

In this section, we describe how to implement Algorithm [T] given the various algorithmic components derived 
in the Section [4] Numerically computing a solution to the Switched System Optimal Control Problem 
defined as in Equation ( 18 1 demands employing some form of discretization. When numerical integration is 
introduced, the original infinite-dimensional optimization problem defined over function spaces is replaced 
by a finite-dimensional discrete-time optimal control problem. Changing the discretization precision results 
in an infinite sequence of such approximating problems. 

Our goal is the construction of an implementable algorithm that generates a sequence of points by 
recursive application that converge to a point that satisfies the optimality condition defined in Equation 



(30). Given a particular choice of discretization precision, at a high level, our algorithm solves a finite 
dimensional optimization problem and terminates its operation when a discretization improvement test is 
satisfied. At this point, a finer discretization precision is chosen, and the whole process is repeated, using 
the last iterate, obtained with the coarser discretization precision as a "warm start." 

In this section, we begin by describing our discretization strategy, which allows us to define our discretized 
optimization spaces. Next, we describe how to construct discretized trajectories, cost, constraints, and 
optimal control problems. This allows us to define a discretized optimality function, and a notion of consistent 
approximation between the optimality function and its discretized counterpart. We conclude by constructing 
our numerically implementable optimal control algorithm for constrained switched systems. 

5.1 Discretized Optimization Space 

To define our discretization strategy, for any positive integer N we first define the N-th switching time space 
as: 

Tn = |(r ,...,r fc ) C [0,1] | = r <n < ■■■ <T k = 1, In-Ti-xl < i Vt G {l,...,fc}|, (142) 

i.e. 1~n is the collection of finite partitions of [0,1] whose samples have a maximum distance of Jj^. For 
notational convenience, given t S Tn, we define |r| as the cardinality of r. Importantly, notice that the sets 
Tn are nested, i.e. for each N € N, Tn+i C Tn- 
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We utilize the switching time spaces to define a sequence of finite dimensional subspaces of X p and X r . 
Given N € N, t e T/v, and fc € {0, . . . , \t\ — 1}, we define ir T> k ■ [0, 1] — > K that scales the discretization: 

/,x J 1 if *€ [r fe ,r fe+ i), 
7r T ,fe(t) = < . (143) 

I otherwise. 

Using this definition, we define T> r p , a subspace of the discrete input space, as: 

f |r|-l ] 

P T , p = | d e V p I d = J2 d ^r,k, dk G E| Vfc ^ . (144) 

Similarly, we define 2? r . r , a subspace of the relaxed discrete input space, as: 

f |T| 1 1 

2? T , r = j d G V r \ d =J2 ^.k, dk G E? Vfc | . (145) 

Finally, we define W r , a subspace of the continuous input space, as: 

[ |r|-l ) 

U T = <ueU\u= UkTt T ,k, Uk € f7 Vfc > . (146) 
I J 

Now, we can define the N-th discretized pure optimization space induced by switching vector r as X rp = W T x 
2? TiP , and the N-th discretized relaxed optimization space induced by switching vector r as AV !r = U T x T> T T . 
Similarly, we define a subspace of X : 

( \r\-l |t|-1 _ \ 

X T = I (u, d) G X | u = u k Tr T , k , u k G M m Vfc, and d = ^ d fc7 r T , fe , d fc £ 1' Vfc 1 . (147) 

[ fc=0 fc=0 J 

In order for these discretized optimization spaces to be useful, we need to know to show that we can use a 
sequence of functions belonging to these finite-dimensional subspaces to approximate any infinite dimensional 
function. The following lemma proves this result and validates our choice of discretized spaces: 

Lemma 19. Let {Tk}k£N with Tk G 71- ■ 

(1) For each £ € X p there exists a sequence {£fc}/ceN. with £/. € X Tk P , such that £ k — > £ as fc — > oo. 

(2) For eac/i (€^ r £/iere exists a sequence {£/c}fceN; urai/i € ^tV^.r; smc/i £/iat — > £ as k — > oo. 

Proof. We only present an outline of the proof, since the argument is outside the scope of this paper. First, 
every Lebesgue measurable set in [0, 1] can be arbitrarily approximated by intervals (Theorem 2.40 in |13|). 
Second, the sequence of partitions {rfc}fc S N can clearly approximate any interval. Finally, the result follows 
since every measurable function can be approximated in the L 2 -norm by integrable simple functions, which 
are the finite linear combination of indicator functions defined on Borel sets (Theorem 2.10 in |13|). □ 

5.2 Discretized Trajectories, Cost, Constraint, and Optimal Control Problem 

For a positive integer N, given a switching vector, r G 7~n, a relaxed control £ = (u, d) € X T r , and an initial 

condition .To G R™, the discrete dynamics, denoted by {^H T fc)}feln c ^ n > are computed via the Forward 
Euler Integration Formula: 

Z^(Tk +1 )=Z^(Tk) + (T k+1 -T k )f(Tk,Z^(Tk),u(T k ),d(T k )), Vfc G {0, ... , |t| — 1}, $\0) = X . (148) 
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Employing these discrete dynamics we can define the discretized trajectory, ■ [0, 1] — > R", by performing 
linear interpolation over the discrete dynamics: 

f V r k+1 -r k ) 



where -k t ^ are as defined in Equation (143 1. Note that the definition in Equation (149) is valid even if 
Tk = Tfe+i for some k G {0, . . . , |t|}, which becomes clear after replacing Equation (148) in Equation (1491. 
For notational convenience, we suppress the dependence on r in when it is clear in context. 

Employing the trajectory computed via Euler integration, we define the discretized cost function, J T : 
X T r — y M: 

Jr(0 = M z(O (l))- (150) 

Similarly, we define the discretized constraint function, %p T : X Tr — > R: 

* r (0= max fcffc^fa)). (151) 

3&J, fee{0,...,|r|} JK " 

Note that these definitions extend easily to points belonging to X T ,p- 

As we did in Section |2.3[ we now introduce some additional notation to ensure the clarity of the ensuing 
analysis. First, for any positive integer N and r £ T/v, we define the discretized flow of the system, <fi T _ t : 
X r -> E" for each t G [0, 1] as: 

^, t (0 = 4°(*)- (152) 

Second, for any positive integer N and t G 7tv, wc define component constraint functions, ip T ,j,t ■ X r —>M. 
for each t G [0, 1] and each j G J as: 

^rj,t(0 = ^(^r,t(0)- (153) 

Notice that the discretized cost function and the discretized constraint function become 

Jr(O = h (<f>r,i(O), and * r (0= max ^(0, (154) 

jej", fce{o,...,|r|} 

respectively. This notation change is made to emphasize the dependence on £. 



5.3 Local Minimizers and a Discretized Optimality Condition 

Before proceeding further, we make an observation that dictates the construction of our implementable 
algorithm. Recall how we employ directional derivatives and Theorem [3] in order to construct a necessary 
condition for optimality for the Switched System Optimal Control Problem. In particular, if at a particular 
point belonging to the pure optimization space the appropriate directional derivatives are negative, then the 
point is not a local minimizer of the Relaxed Switched System Optimal Control Problem. An application 
of Theorem [3] to this point proves that it is not a local minimizer of the Switched System Optimal Control 
Problem. 

Proceeding in a similar fashion, for any positive integer N G N and r G T/v, we can define a Discretized 
Relaxed Switched System Optimal Control Problem: 

Discretized Relaxed Switched System Optimal Control Problem. 

min {J r (0|* r (0<0}. (155) 
The local minimizers of this problem are then defined as follows: 
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Definition 6. Fix N € N, and r G Tn ■ Let us denote an s-ball in the X-norm centered at £ induced by 
switching vector r by: 

M-,*(£,e) = {fe* r , r | \\Z-Z\\ X <£}. (156) 
We say that a point £ S X rr is a local minimizer of the Relaxed Switched System Optimal Control Problem 



Induced by Switching Vector r defined in Equation (155) if ^riO < and there exists e > such that 



Mt) > MO each £ e AT T ,x^ e ) n X r,r I *r® < 0}. 

Given this definition, a first order numerical optimal control scheme can exploit the vector space structure 
of the discretized relaxed optimization space in order to define discretized directional derivatives that find 
local minimizers for this Discretized Relaxed Switched System Optimal Control Problem. Just as in Section 



3.1 we can employ a first order approximation argument and the existence of the directional derivative of 
the cost, DJ T (proven in Lemma 30), and of each of the component constraints, Dij} Ti j jTh (proven in Lemma 



32 ), for each j & J and k € {0, . . . , \r\} in order to elucidate this fact. 

Employing these directional derivatives, we can define a discretized optimality function. Fixing a positive 
integer N and t £ Tzv, we define a discretized optimality function, 9 T : X Tp — > (— oo, 0] and a corresponding 
discretized descent direction, g T : X Tp — > X Tr : 

T (O= min Cr(f,0» s r (0 = argminCr(£,O> ( 157 ) 

where 

|max{DJ T (£;£'-£), max Bfaj^fc - + 7*t(o) + IIC ~ £11* if* T (0<0, 

^ t (^ _ J I fcG{0,...,|r|} J 

lmaxjDJ^^'-O-*^). m P , , DVr,i,r,te^-Oi + llC'-^lk if*r(0>0, 
I, L je^r, fcG{o,...,|r|} J 

(158) 

and 7 > is a design parameter as in the original optimality function 0, defined in Equation (30). Before 
proceeding, we make two observations. First, note that T (£) < for each £ € <%V !?) , since we can always 
choose £' = £ which leaves the trajectory unmodified. Second, note that at a point £ g AV iP the directional 
derivatives in the optimality function consider directions £' — £ with £' G AV !r in order to ensure that first 
order approximations belong to the discretized relaxed optimization space X T r which is convex (e.g. for 
< A< 1, J T (0 + ADJ T (ft£'-0 « J r ((l - A)£ + Af') where (1 - A)£ + A£' e ^ T , r ). 

Just as we argued in the infinite dimensional case, we can prove, as we do in Theorem \12\ that if 
T (£) < for some £ g ^r.pj then £ is not a local minimizer of the Discretized Relaxed Switched System 
Optimal Control Problem. Proceeding as we did in Section [3j we can attempt to apply Theorem [3] to prove 
that 6 encodes local minimizers by employing the weak topology over the discretized pure optimization space. 
Unfortunately, Theorem [3] does not prove that the element in the pure optimization space, £ p € X p , that 
approximates a particular relaxed control £ r € X T ^ r C X r at a particular quality of approximation e > with 
respect to the trajectory of the switched system, belongs to X T p . Though the point in the pure optimization 
space that approximates a particular discretized relaxed control at a particular quality of approximation 
exists, it may exist at a different discretization precision. 

This deficiency of Theorem [3] which is shared by our extension to it, Theorem [6j means that our com- 
putationally tractable algorithm, in contrast to our conceptual algorithm, requires an additional step where 
the discretization precision is allowed to improve. Nevertheless, if we prove that the Discretized Switched 
System Optimal Control Problem consistently approximates the Switched System Optimal Control Problem 
in a manner that is formalized next, then an algorithm that generates a sequence of points by recursive 
application that converge to a point that is a zero of the discretized optimality function is also converging 
to a point that is a zero of the original optimality function. 

Formally, motivated by the approach taken in [29], we define consistent approximation as: 

Definition 7 (Definition 3.3.6 |29J. The Discretized Relaxed Switched System Optimal Control Problem as 
defined in Equation ( |155[ ) is a consistent approximation of the Switched System Optimal Control Problem 
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as defined in Equation (18) if for any infinite sequence {rjjigN and {£j}ieN such that T; G 7i and £j € <%V;,p 
/or each i € N, i/ien lim^oo |0 Ti (£i) — $(£i)l = 0, where 9 is as defined in Equation (30) and 6 T is as defined 
in Equation (157). 

Importantly, if this notion of consistent approximation is satisfied, then a critical result follows: 

Theorem 10. Suppose the Discretized Relaxed Switched System Optimal Control Problem, as defined in 
Equation (155), is a consistent approximation, as in Definition^ of the Switched System Optimal Control 
Problem, as defined in Equation ( |18[ ). Let {r{\i^ and {£j}ieN be such that Ti G T and £j € X TuP for each 
i £ N. In this case, j/linii_>. 00 9 Ti (£,i) = ; then lim^oo = 0. 



Proof. Arguing by contradiction, suppose there exists a S > such that liminfj 
super-additivity of the liminf, 

liminf Ti (&) - liminf 0(&) < liminf T< (&) - 0(&). 

Rearranging terms and applying Definition [7j we have that: 

liminf 9 Ti (&) < liminf (0 Ti (&) - 0(&)) + liminf 0(&) < -5. 



0(&) < -(5. Then by the 
(159) 

(160) 



which contradicts the fact that lim^oo 9 Ti (£i) = 0. Since by Condition [l] in Theorem [5j liminf^oo 0(£,) < 
limsup^^ 0(£i) < 0, we have our result. □ 

To appreciate the importance of this result, observe that if we prove that the Discretized Relaxed Switched 
System Optimal Control Problem is a consistent approximation of the Switched System Optimal Control 
Problem, as we do in Theorem 13 and devise an algorithm for the Discretized Relaxed Switched System 
Optimal Control Problem that generates a sequence of discretized points that converge to a point that is 
a zero of the discretized optimality function, then the sequence of points generated actually converges to 
a point that also satisfies the necessary condition for optimality for the Switched System Optimal Control 
Problem. 



5.4 Choosing a Discretized Step Size and Projecting the Discretized Relaxed 
Discrete Input 

Before describing the step in our algorithm where the discretization precision is allowed to increase, we 
describe how the descent direction can be exploited in order to construct a point in the discretized relaxed 
optimization space that either reduces the cost (if the original point is feasible) or the infeasibility (if the 
original point is infeasible). Just as we did in Section 3.4 we employ a line search algorithm similar to 
the traditional Armijo algorithm used during finite dimensional optimization in order to choose a step size 
(Algorithm Model 1.2.23 in (53]). Given N € N, t e Tn, a e (0, 1), and /3 € (0, 1), a step size for a point 
£ G %t,p<i is chosen by solving the following optimization problem: 

dn{fc e N | J T (C + (3 k (g T (0 ~ 0) - MO < ^(3 k 9 T (0, 

MO = { *r(£ + PHaAO - 0) < a[3 k e T (0} if *r(£) < 0: ( 161 ) 

dn{fc GN|$ T (? + I3 k (9r(0 - 0) - *r(0 < a0 k OAO} if *A0 > 0- 



Continuing as we did in Section 13.41 given N € N we can apply Fn defined in Equation (34) and V 



N 

defined in Equation (35) to the constructed discretized relaxed discrete input. The pulse width modulation 
at a particular frequency induces a partition in Tn according to the times at which the constructed pure 
discrete input switched. That is, let on ■ X r — > Tn be defined by 

^(«,d) = {o}u|4 + wE^( d )]ifw) \ ■ ( 162 ) 

,<?} 

.,2 N -1} 
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Employing this induced partition, we can be more explicit about the range of by stating that Pn(Q G 
X(t n (C),p for each ^ £ X r . 

Now, given given N £ N, r £ T/v, <5 G (0, oo), /3 g (^tji lj 1 w G (0, 1), and /c max € N, a frequency at which 
to perform pulse width modulation for a point £ € AV. P is computed by solving the following optimization 
problem: : 

'min{fc < fc max \Z'=Z + /3^(g T (0 ~ £), afi k < (1 - w)a/3M£), 

v T (£,fc max ) = { 0>*(O) ^ M" T(S) if *r(0 < 0, (163) 

linjfc < fc max |£'=£ + /3"*«> ( 5r (0 - 0, ap k < (1 - w) tt ^«), 

"*r(0 < - aP k )0 T (£)} if *r(0 > 0. 

In the discrete case, as opposed to the original infinite dimensional algorithm, due to the aforementioned 
shortcomings of Theorem [3] and |6j there is no guarantee that the optimization problem solved in order to 
v T is feasible. Without loss of generality, we say that v T {Q = 00 f° r each £ £ X T ^ r when there is no feasible 



solution. Importantly letting Nq £ N, tq £ Tn , and £ £ X T>r , we prove, in Lemma 40 that if < then 
for each rj £ N there exists a finite N > Nq such that v rTN ^)(^, N + 77) is finite. That is, if < 0, then v T 
is always finite after a certain discretization quality is reached. 

5.5 An Implementable Switched System Optimal Control Algorithm 

Consolidating our definitions, Algorithm [2] describes our numerical method to solve the Switched System 
Optimal Control Problem. Notice that note that at each step of Algorithm [2j £j £ X TjtP . Also, observe the 
two principal differences between Algorithm [I] and Algorithm [2j 

First, as discussed earlier, v T maybe infinite as is checked in Line [TO] of Algorithm [5] at which point the 
discretization precision is increased since we know that if < 0, then v T is always finite after a certain 
discretization quality is reached. Second, notice that if 8 T comes close to zero as is checked in Line [3] of 
Algorithm [2j the discretization precision is increased. To understand why this additional check is required, 
remember that our goal in this paper is the construction of an implementable algorithm that constructs a 
sequence of points by recursive application that converges to a point that satisfies the optimality condition. 
In particular, 8 T may come arbitrarily close to zero due to a particular discretization precision that limits the 
potential descent directions to search amongst, rather than because it is actually close to a local minimizer 
of the Switched System Optimal Control Problem. This additional step that improves the discretization 
precision is included in Algorithm [2] to guard against this possibility. 

With regards to actual numerical implementation, we make two additional comments. First, a stopping 
criterion is chosen that terminates the operation of the algorithm if 9 T is too large. We describe our selection 
of this parameter in Section [7] Second, due to the definitions of D J T and Dip T j Tk for each j £ J and 
k £ {0, . . . , |t|}, the optimization required to solve 9 T is a quadratic program. 

For analysis purposes, we define r r : {£ £ X TyP | z/ T (£, fc max ) < 00} — > X p by: 

rV(£) =Pv A ^M + ^ AO (9r{0-(i)). (164) 

We say {^IjeN is a sequence generated by Algorithm^if = F Tj (£j) for each j £ N. We can prove 
several important properties about the sequence generated by Algorithm[2] First, letting {Ai}i S N, {Ti}«eN, 
and {£i}igN be the sequences produced by Algorithm [2j then, as we prove in Lemma 42 there exists i £ N 



such that, if * Tio (£iJ < °> then < and * Ti (£i) < for each i > i . That is, once Algorithm [2] 

finds a feasible point, every point generated after it remains feasible. Second, as we prove in Theorem |14J 
limj-j.oo 9(£j) — for a sequence {£j}jeN generated by Algorithm[2j or Algorithm [2] converges to a point that 
satisfies the optimality condition. 
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Algorithm 2 Numerically Tractable Algorithm for the Switched System Optimal Control Problem 
Require: jV G N, r G T No , Co G ^V 0)P , a € (0,1), a € (0,oo), /? G (0,1), € (75,1), 7 G (0,oo), rj G N, 

AG(o,oo),xe(o,i),we(o,i). 

l: Set j = 0. 

2: Compute T -(^j) as defined in Equation (1571. 
if Tj (&) > -A2"^ then 

Set = £n ^+1 = + !, = oty+ifo)- 
Replace j by j + 1 and go to Line [2] 
end if 

Compute g Tj ((,j) as defined in Equation ( |157[) . 
Compute l^rACj) as defined in Equation (161). 
Compute v r . (£j , Nj + rf) as defined in Equation ( |163 ) . 
if i/ T . 2Vj + 77) = 00 then 

Set Cj+i = £7, ^+1 = + t j+i = cr w J+ i(^+i)- 
Replace j by j + 1 and go to Line [2] 
end if 

Set O+i = /V,(e 3 ',2V j+J7 )fe + ^ ( ^W&) - &))> ^i+i = max {^,^(0,^+7,)}, r J+1 = 

0'jV i+ i($7+l)< 

15: Replace j by j + 1 and go to Linc[2j 
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6 Implement able Algorithm Analysis 

In this section, we derive the various components of Algorithm [2] and prove that Algorithm [2] converges to a 
point that satisfies our optimality condition. Our argument proceeds as follows: first, we prove the continuity 
and convergence of the discretized state, cost, and constraints to their infinite dimensional analogues; second, 
we construct the components of the optimality function and prove the convergence of these discretized 
components to their infinite dimensional analogues; finally, we prove the convergence of our algorithm. 

6.1 Continuity and Convergence of the Discretized Components 

In this subsection, we prove the continuity and convergence of the discretized state, cost, and constraint. 
We begin by proving the boundedness of the linear interpolation of the Euler Integration scheme: 

Lemma 20. There exists a constant C > such that for each N G N, r G Tn, £ G X T r , and t G [0, 1], 

\\z (0 {t)\\ 2 <C (165) 

Proof. We begin by showing the result for each t G r. By Condition [l] in Assumption [l] together with the 
boundedness of ||/(0, Xq, 0, 6j)||2 for each i G Q, there exists a constant K > such that, for each N G N, 
t G Tn, £ G X T , r , i G Q, and k G {0, . . . , |r|}, 

||/(t*, « (c) (7)fc), «(**)> e*)|| 2 ^ ^(ll z(C) M| 2 + l)- (166) 



Employing Equation (148) and the Discrete Bellman-Gronwall Inequality (Exercise 5.6.14 in [15]), we h ave: 

k q 

||* {e fa)|| 3 < IMI2 + ^^||/fa,z(«)(T,),u(r,),e 4 )|| 2 



2 1 

j=0 i=l 



< (IM2 + I) [1+ 2 , 
<e* K (\\x Q \\ 2 + l), 



qK ^ N ( 167 ) 
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thus obtaining the desired result for each t £ r. 

The result for each t £ [0,1] follows after observing that, in Equation (1491, {^ T ^~^- Tk ) < 1 f° r each 
t £ [r fc , T fe+ i) and k £ {0, . . . , |r|}. 



□ 



In fact, this implies that the dynamics, cost, constraints, and their derivatives are all bounded: 



Corollary 13. There exists a constant C > such that for each N £ N, r £ 1~n, j € J , and £ = (u, d) £ 



(1) \\f{t,z^(t),u(t),d(t))\\ 2 <C, m,zM(t),u(t),d(t)) 



i',2 



<c, 



< C. 



i.2 



(2) \h (z^(t))\ <C, ||^(*«>(t)) 

(3) \h,izW(t))\<C, d £(z^(t)) 



<C, 



< c. 



Proof. The result follows immediately from the continuity of /, ftoj 75?) ^j, and for each j £ J ', 

as stated in Assumptions [T] and [2] and the fact that the arguments of these functions can be constrained to 
a compact domain, which follows from Lemma 20 and the compactness of U and E^. □ 

Next, we prove that the mapping from the discretized relaxed optimization space to the discretized 
trajectory is Lipschitz: 

Lemma 21. There exists a constant L > such that, for each N £ N, t £ Tn, £1,62 G <^V,r and t £ [0, 1]: 

UtMi) - M6)||a ^ - (168) 
where 4> T .t(£) is as defined in Equation ( 152[ ). 

Proof. We first prove this result for each t £ t. For notational convenience we will define Atj = 7^4.1 — tj. 
Letting £1 = (m, di) and £2 = (U2, d 2 ), notice that for j £ {0, . . . , \t\ — 1}, by Equation ( 148 ) and rearranging 
the terms, there exists L' > such that: 



\<f>T,T j + 1 (€l) ~ 0T,T i+ l(&)|| 2 - || ^T.T-,- (6) - ^T.TJ (6) 1 1 2 < 

< At,- I j / (73 , t ,t 3 . (£1) , tii (rj) , di (tj)) - / , ^ TjT;f (£ 2 ) , w 2 (rj) , d 2 (Tj)) || 2 

^ 2wlkr,r,(Cl) -^(fojUa + L'AT J (||ui(T i )- 1*2(^)112 + \\dl(Tj) - d 2 (Tj)\\ 2 ) , 



(169) 



where the last inequality holds since the vector field / is Lipschitz in all of its arguments, as shown in the 
proof of Theorem J2J and Ar, < ^jv by definition of Tn- 

Summing the inequality in Equation (169) for j £ {0, . . . , k — 1} and noting that 4> t .t (£i) — ^t.to (C2): 



V 



k-l 



k-l 



(&) - <W6)|| 2 ||<W6) - <W6)|| 2 + L 'Y, At j IM t ;) - U ^W 



3=0 



3=0 



k-l 



L'Ys&TjWd^-d^Tj)^. (170) 
3=0 
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Using the Discrete Bellman-Gronwall Inequality (Exercise 5.6.14 in [291) and the fact that ^1 + ^ 2 " < e L , 

'\r\-l \r\-l 

,(6) - &™(6)ll a < ^ ( E At o ~ Mtj)\\ 2 + E Ar J H d i( r i) - d 2(r,-)|| 

J=0 j=0 



|r|-l 



\ E ll U l( T i) " U 2(r. 

\ o=0 



2 

J / 1 1 2 



|r|-l 



\ j=0 



(171) 



£||6-6IU, 



where L = L'e L , and we employed Jensen's Inequality (Equation A. 2 in 26J) together the fact that the 
A'-norm of £ G X T . r can be written as a finite sum. 

The result for any t £ [0, 1] follows by noting that </> T ,t(0 is a convex combination of <fi T . Tk (£) and 4> T , Tk+1 (£) 
for some k € {0, . . . , |r| - 1}. □ 

As a consequence, we immediately have the following results: 

Corollary 14. There exists a constant L > such that for each N 6 N, r 6 T/v, 6 = ( u ij^i) G AV.r; 
6 = ("2, d 2 ) € # T ,r did i € [0, 1]: 

(i) ||/(t, T)t (a), «x (*), dx(t)) - f(t, faA&MQMt)) lis < 

< £(||6 - &||* + |Mt) - ua(t)|| 2 + |Mi(t) - da(t)|| a ), 



(2) 



(3) 



(t,0r,t (6), «1 (*),*(*)) - ^(t,4>r,t{^)Mt),d 2 {t)) 



< 



i,2 



< - 611* + hx(t) - u 2 (t)\\ 2 + - daWlla). 



9f <9f 



du 



< 



< £(||6 - 611* + HW - « 2 WII 2 + ||di(t) - d 2 (t)|| : 



where 4> r ,t(Q is as defined in Equation (152 1. 



Proof. The proof of Condition [T] follows by the fact that the vector field / is Lipschitz in all its arguments, 
as shown in the proof of Theorem [2] and applying Lemma 21 The remaining conditions follow in a similar 
fashion. □ 

Corollary 15. There exists a constant L > such that for each N E N, t E Tn, 6 = ( u i,di) G X r ,T, 
6 = ( u 2, da) € X r , T , j e J, and t <G [0, 1]; 

(1) |/i (<M6))-M<M6))| < L 116-611*, 

(2) 1^(^1(6)) - t?(<M6))| 2 < £ 116 -611*, 

(3) |M<M6)) -M<M6)) I <£||6- 611*, 

(4) ||^(^(6)) -^(^ t (6))|| 2 <£||6-6ll* 3 

where <f> T ,t(Q is &s defined in Equation ( |152[ ). 

Proof. This result follows by Assumption [2] and Lemma [21] □ 
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Even though it is a straightforward consequence of Condition [T] in Corollary [l5j we write the following 
result to stress its importance. 

Corollary 16. Let N G N and t € Tzv, then there exists a constant L > such that, for each ^1,^2 G X rr : 

\Mtl)- M&)\ <i||6-6lU (172) 



where J T is as defined in Equation ( 150 1 . 

In fact, \I/ T is also Lipschitz continuous: 
Lemma 22. Lei N G N and r G T/V; iften i/iere exists a constant L > swc/i t/ia£ ; /or eac/i ^1,^2 G AV: 

|*r(€l) " *r(6)l <£ (173) 



where ^ T is as defined in Equation (151). 

Proof. Since the maximum in \I/ T is taken over J x k G {0,...,|r|}, which is compact, and the maps 
(j,k) 1 — y ^V,i,Tis(£) are continuous for each £ G X T , we know from Condition [3] in Corollary 15 that there 
exists L > such that, 



*r(6)-*r(6 



(i,fc)eJ r x{o,...,|r|} 



VV,j,Tfc(£l) 



(j,k)£jx{0,...,\T\} 



-t-u^ I n (6) " VV.j.r* (6) 



(174) 



<£||fc-6IU- 

By reversing £1 and £2, and applying the same argument we get the desired result. 

We can now show the rate of convergence of the Euler Integration scheme: 
Lemma 23. There exists a constant B > such that for each N G N, r G Tn, £ G X r r , and t G [0, 1] 

B 



□ 



z^(t)-x^>(t)\L < 



2 — 2^ : 



(175) 



where x^ is the solution to Differential Equation (13 1 and Zr is as defined in Difference Equation (149 1. 

Proof. Let £ = (u, d), and recall that the vector field / is Lipschitz continuous in all its arguments, as shown 
in the proof of Theorem [2j By applying Picard's Lemma (Lemma 5.6.3 in [22]), we have: 



z«>(t)-xl«(t)|L < e 



(Of 







ds 



<Le L (l+\\f(r k ,z^(r k ),u(T k ),d(T k ))\\ 2 ) \ 

j |r|-l 

< — Le L (l + C) ]T(r fc+1 



(a) -/(s, u(«),d(s)) ds 

2 

/(r fc ,z(«)(r fe ), U (r fe ),d(r fe )) + 
-f(s,z^( Tk ) + "~ Tfc (s«Wi) - *<«(7*)),ufa),d(7*) 



ds 



Is - r fc |ds 



fc=0 



B 

2N> 



(176) 

where C > is as defined in Condition 111 in Corollary 13 and, B = (1 + C)Le L , and we used the fact that 



Tfc+i — Tfc < T^r by definition of Tn in Equation ( 142 ). 



□ 
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Importantly we can show that we can bound the rate of convergence of this discretized cost function. We 
omit the proof since if follows easily using Assumption [2] and Lemma 23 



Lemma 24. There exists a constant B > such that for each N G N, r € T/v. and £ G X Tf , 

B 



\MO-m\< 



2 N ' 



(177) 



where J is as defined in Equation (15) and J T is as defined in Equation (150). 

Similarly, we can bound the rate of convergence of this discretized constraint function. 
Lemma 25. There exists a constant B > such that for each N G N, r G T/v > o,nd £ G X r ,r 

B 



|*r(0"*(OI< 



2 N '' 



(178) 



where ^> is as defined in Equation ( 16 I and ^ T is as defined in Equation ( 151 1 . 



Proof. Let C > be as defined in Condition [T] in Corollary [TJ and let L > be the Lipschitz constant as 
specified in Assumption [2j Then, using the definition of Tn hi Equation ( 142 1, for each k G {0, . . . , |t| — 1} 

and t G [Tfc,T fe+ i], 



h^x&it)) -h^x^in))] < L / \\f(s,x^(s),u(s),d(s))\\ 2 ds< 



LC 



(179) 



Moreover, Condition [3] in Assumption [2] together Lemma 23 imply the existence of a constant K > such 
that: 

M^W)-M* K) W)l4' ( 18 °) 



Employing the Triangular Inequality on the two previous inequalities, we know there exists a constant B > 
such that, for each t G [it, t^+i], 



< 



B 



2 N- 



Ih^x^^-h^z^iru)) 
Let t' G argmax tS [ 01 ] hj(x^\t)), and let k(£') G {0, . . . , |r| — 1} such that t' G [T re (t'), T K ( t /) +1 ] . Then 

max A(a^(t)) - max h 3 (r k )) < h s (x® &)) - hj{z® {t k{V ))) < V 
*e[o,ij ke{o,...,\T\} 

Similarly if k' G argmax fce { | r |} hj(z^> (rk)) , then 



(181) 



2 N- 



(182) 



fcg maX T|} ^-(z(«)(r fc )) - max hj(x^(t)) < h 3 (z^ {r k ,)) - h,(x^ (r k ,)) < 



2 N- 



This implies that: 



*(£) - 1f T (£) < max ( max hJx^Ut)) - max /i 7 - (z (5) (r k )) ) < 
K ' y '-jej\te[o,i] n w ' fce{o,..,|r|} n V >>) ~ 

* (£) - *(£) < max ( max ft,,- (z^Htu)) - max /i 7 -(a; K) (i)) 
v ; V ' ~ jeJ \ke{o,...,\r\} n v 7; te[o,i] n w ' 



w 

B 

W 



which proves the desired result. 



(183) 

(184) 
(185) 

□ 
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6.2 Derivation of the Implementable Algorithm Terms 

Next, we formally derive the components of the discretized optimality function, prove the well-posedness of 
the discretized optimality function, and prove the convergence of the discretized optimality function to the 
optimality function. We begin by deriving the equivalent of Lemma [4] for our discretized formulation. 

Lemma 26. Let N G N, r € Tn, £ = fae?) G X T .r, C = (u 1 ,d!) € X T , and Tjt be as defined in Equation 
(152|. Then, for each k £ {0, . . . , |t|}, the directional derivative of <p TtTk , as defined in Equation (25), is 
given by 



k-l 



j=0 



D0r,r fc (£;O = Efa + 1 - T j)^r\r k ,T j+1 )[^(T j ,<P TjTj (0,u{T j ),d(T j ))u , (T j ) + 



du 



+ E ffa'tr* (£), ufa), e^fa) , (186) 
i=i ' 

where fa j T j ) is unique solution of the following matrix difference equation: 

= 4°fa»Ti) + fa+i " ^)|^fa,0 T , Tt (O,«fa),dfa))$(«(r fe ,r,), ^fa.Tj-) = J, (187) 
for each k € {0, . . . , |r| — 1}. 

Proof. For notational convenience, let z^* 1 = z(?+ A £ ), = u + Xvf, and d^' = d+ Xd'. Also, let us define 
Az (A) = z {\) _ thuSj for each k£{0,..., |r|}, 

fe-l 



A^>(r fc )=Efa + i-^(/fa,* (A ^ 

= Efa+i-^)f A E^fa)/( r ^ z(A) fa)' M(A) fa)' e O+ 



J=0 



(188) 



+ gfa,z(«)(r J ) + ^ J Az( A )(r J ), U (A) fa),dfa))Az( A '(r J )+ 

+ A |{fa^ (0 fa)> u fa) + ^ujA u 'fa)^fa)) M 'fa)^ . 

where {/',.,[/ „ C [0, 1] and {Vu,j} l ?l C [0, 1]. 

Let {j/fa)}|lj. be recursively defined as follows: 

2/fa+l) = 2/fa) + fa+l - 7ft) I -^(T k ,Z (i \T k ),u(T k ),d(T k ))y(T k ) + —(T k ,Z ( - i) (T k ),u(T k ),d(T k ))u'(T k ) + 



+ E d ifa)/fa^ K) fa) ! ufa),e i ) , 2/(r ) = 0. (189) 



We want to show that Azi ^ Tk ^ — > y{T k ) as A 1 0. Consider: 



3/, 



Tfc, 



* K) fa), l*fa), dfa))yfa) - ^ fa, z«)(r fc ) + ^AzW(r t ) lt( W(T t ), d(r fc )) 



< L 



2/fa) - 



< 



AzW(r fe ) 



+ L (|| AzW(r fc )|| 2 + A |Kfa)ll 2 ) ||y(r fe )|| 2 , (190) 
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which follows by Assumption [T] and the Triangular Inequality. Also, 



^ ' (T k ,z^(T k ),u(T k ),d(T k )) - ^(T k ,z^\T k ),u{r k ) + u u>k Aw' (r k ) , d(r k )) ) u'(r k ) 



du 



<LX\\u'{r k )\\i. 



(191) 



and 



<L||Az( A )(r fc )|| 2 + iA|| U '(r fc )|| 2 . 

(192) 



E d '^) (f(rk,z^(u)Mrk),ei) ~ f(r k , z^(r k ), uWfa), *)) 

i=l 

Hence, using the Discrete Bellman-Gronwall Inequality (Lemma 5.6.14 in [23]) and the inequalities above, 
AzW(r k ) 



y{Tk) 



A 



fc-1 



3=0 



< Le L £> J+1 - Tj ) ||AzW(r,)|| 2 + A || U '(r,-)|| 2 bfo)^ + 



goes to zero as A 1 0, thus obtaining that 



where we used the fact that (1 + 2JY < e . But, by Lemma |2l| the right-hand side of Equation (|193[) 



+ A Wu'^t + || AzW^OH, + A Ih/^)!^ (193) 



(194) 



The result of the first part of the Lemma is obtained after noting that Dcf> T}Tk (£; £') is equal to y{r k ) for each 

/•• {" ■ !• 

□ 

Next, we prove that D0 T r( . is bounded by proving that $^ is bounded: 
Corollary 17. There exists a constant C > swc/i </iaf /or eac/i TV G N, r G T/v ; £ G Xrr> and k,l G 



H°(r k , 



n) 



1 1,2 



<c, 



where {r k ,Ti) is £/ie solution to the Difference Equation ( 187[ ). 

Proof. This follows directly from Equation (187) and Condition [l] in Corollary 13 



(195) 



□ 



Corollary 18. There exists a constant C > such that for each N G N, r G T/v, £ G AV,r; C G ^t, 
fcG{0,...,|r|}: 

iiD^, Tfc (c;r)ii 2 <ciiriu, age) 



where D</> T T( . (£; £') is as defined in Equation (186). 

Proof. This follows by the Cauchy-Schwartz Inequality together with Corollary [13] and Corollary [17] □ 
We now show that $r is in fact Lipschitz continuous. 

Lemma 27. There exists a constant L > smc/i that for each N G N, r G 7~/v, £i,£2 G AV r , <f&d fc 5 ^ G 
{0,...,|r|}; 

QpHw) - QpHw) .„<i||ei-6L. ( 197 ) 



i.2 



where * s solution to Difference Equation (187). 
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Proof. Let £1 = (u\,di) and £2 = (^2,^2)- Then, using the Triangular Inequality: 



4 ei) ^)-$£ 2) (Tfc,n) 



< 



fe-1 



i=0 



<9x 



/,2 



<9x 



2 / 

(198) 

The result follows by applying Condition [T] in Corollary [T3| Condition [2] in Corollary [14} the same argument 
used in Equation (43), and the Discrete Bellman-Gronwall Inequality (Exercise 5.6.14 in Q 

A simple extension of our previous argument shows that D</> r Tk (£, •) is Lipschitz continuous with respect 
to its point of evaluation, £. 

Lemma 28. There exists a constant L > such that for each N £ N, r £ T/v, £i)£2 € <^T,rj £' € <%V, and 
fcG{0,...,|r|}: 

l|D^,r fc (6;n-D^r fc (6;nil2<i|ia-e2||^||e'||^, (199) 



where Dtf> TtTk is as defined in Equation (186 1. 

Proof. Let £1 = (ui,di), £2 = ("2,^2), and £' = (v! ,d'). Then, applying the Triangular Inequality: 



||D0T,rja;O-D<W6;O| 



< 



fc-i 



9/ 



(.2 



9/ 



(Tj , Z^ (Tj ) , Ui {Tj) , d x (Tj-)) 



||u'(t j )|| 2 + 



i.2 



IK(Ti)|| 2 + 



i.2 



E 4 ei) (Tfe,r j+1 ) - 4^(r fe ,T j+1 ) /(r,,^)^),^^),^) |4(r, 

* — 1 1 %,2 %,2 



£=1 



+ U^(r k ,r j+1 ) ffaz^ir&u^T&ei) - /(^^(r,-),^),^) 

II 1,2 %,2 I 

(200) 

The result follows by applying Lemma 27 Corollary 17 Condition [l] in Corollary [T] Conditions [l] and [3] in 
Corollary 14 and the same argument used in Equation (43). □ 

Employing these results, we can prove that D<^ T)T( . (£; £') converges to D</> Tfe (£;£') as the discretization is 
increased: 

Lemma 29. T/iere exists B > swc/i i/iai /or eac/i AT eN, r € 7at, £ € <%V,r, £' € AV and fc g {0, . . . , |r|}; 



||D0 Tji (C;O-D^,r,(C;OII 2 < 



2 w 



(201) 



where D(p Tk and D0 T . T(c are as defined in Equations (48) and (186), respectively. 



Proof. Let £ = (u,d), £' = (u',d'). First, by applying the triangle inequality and noticing that the induced 
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matrix norm is compatible, we have: 



|D^(6O-D0™(£;O|| a < 



k-l 



^E 

j=0 - T i 



T i + i 



$«)(r fc , S )-$(. ?) (^,^+i) 



i.2 



$(€)(r fe ,s) 



ll u '( T J)ll 2 rfs + 



fT.,+1 _9 

j=0 JT 3 i = l 



$^(T fc , S )-$(0( rfejrj . +1 )|| j2 ||/( rj , z («) (Tj);U(rj)ie2 )|| 2 j Id^JIda. 

(202) 

Second, let G {0, . . . , |r|} such that t € [ r «(t)) r «(t)+i] f° r eacn * € [0, 1]. Then, there exists K > 
such, thcit 

- *«> (r K(s) ) || < \\x®(a)-z®(8)\\ + \\z®(8)-z®(T K(a) ) \\ 
< \\x^(s)-z^(s)\\ + ( S -t k{s) )C 
K 



(203) 



< 



2 N-- 



where C > is as in Condition [T] in Corollary |13| and we applied Lemma [23] and the definition of T/v hi 
Equation ( fl"42 ). 

Third, in a fashion similar to how we defined our discretized trajectory in Equation (1491, wc can define 
a discretized state transition matrix, for each k £ {0, . . . , |t|} via linear interpolation on the second 
argument: 



|r|-l 
J=0 



(204) 



where t t j is as defined in Equation (1431. Then there exists a constant K 1 > such that for each t G [0, 1] 



< 



i,2 



< 



$«)(r fe ,i)-$«)(r fe ,i) ^+ $(.«)(r fe ,i)-$(.«)(r fe ,r K(t) ) 
2F' 



/,2 



(205) 



where the last inequality follows by an argument identical to the one used in the proof of Lemma[23j together 
with an argument identical to the one used in Equation ( 203 1 . 

Finally, the result follows from Equation (202) after applying Condition [T] in Corollary 13 Corollary 
ff\ C onditions [T| and [3| in Corollary 14 Equations (203) and (205), and the same argument as in Equation 

q43). □ 

Next, we construct the expression for the directional derivative of the discretized cost function and prove 
that it is Lipschitz continuous. 



Lemma 30. Let N £ N, r £ Tn> £ G Af Tr , £' G X T , and J T be defined as in Equation (150). Then the 
directional derivative of the discretized cost J T in the £' direction is: 



DJ T (£;O = ~(^r,i(0)D^,i(£;O- 



(206) 
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Proof. The result follows using the Chain Rule and Lemma 26 



□ 



Corollary 19. There exists a constant L > such that for each N G N, r G T/v? £i,£2 G <^r,r> £' G <^r ; 

|DJ T (e i; n-DJ T (6;r)l<^lla-6lUlhlU> (207) 



where DJ T is as defined in Equation (206 1. 

Proof. Notice by the Triangular Inequality and the Cauchy Schwartz Inequality: 

dh 



|DJ T (£i;0-DJ r (6;OI< 



dx 



(</V,l(£l)) 



P^-,i(Si;'?)-D0r,i(6;'7)ll 2 + 



|D^ T ,i(6;»7)|| 2 . (208) 



The result then follows by applying Condition [2] in Corollary [T3} Condition [2] in Corollary [l5j Corollary [T8| 
and Lemma [28l □ 

In fact, the discretized cost function converges to the original cost function as the discretization is 
increased: 

Lemma 31. There exists a constant B > such that for each N G N, r G Tn, £ G <%V, r; a^rf £' G AV-' 



|DJ T (e;C')-DJ(?;Ol< 



2 N- 



(209) 



where DJ is as defined in Equation (66) and DJ T is as defined in Equation (206). 
Proof. Notice by the Triangular Inequality and the Cauchy Schwartz Inequality: 



|DJ T (£;0-D./(£;OI < 



Oho 
dx 



(<M0) 



|d<M£;£')-d<M£;£') 



dx 



||D^ Tll (C;Oll a . (210) 



Then the result follows by applying Condition [2] in Assumption [2j Condition [2] in Corollary [T] Lemma 23 
Lemma 29 and Corollary fl8| 



□ 

Next, we construct the expression for the directional derivative of the discretized component functions 
and prove that they are Lipschitz continuous. 



Lemma 32. Let N G N, t G Tn, £ G X Tr , £' G X T , j G J , and ip T ,j,T k be defined as in Equation (153). Ther. 
the directional derivative of each of the discretized component constraints ^pT,j.r k for each k G {0, . . . , |t|} ir. 



the £' direction is: 



3h 



Proof. The result is a direct consequence of the Chain Rule and Lemma |26| 



(211) 

□ 



Corollary 20. There exists a constant L > such that for each N G N, t G Tn, £1,62 G X Tr , £' G X T , and 
fcG{0,...,|r|}: 

iD^-,r fc (a;n-D^ fc (6;ni <mx-&\\A?\\ x , (212) 



where T>tj) T j Tk is as defined in Equation (211). 
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Proof. Notice by the Triangular Inequality and the Cauchy Schwartz Inequality: 

dh~ 



dx 



(<W£i)) 



IP^(e l ;0-D^(6;Oll a + 



^(^(e0)-^(*™(&)) 



IP<W6;OII 2 - (213) 



The result then follows by applying Condition [3] in Corollary [l3j Condition [4] in Corollary 15 Corollary 18 
and Lemma l28l 



□ 

In fact, the discretized component constraint functions converge to the original component constraint 
function as the discretization is increased: 

Lemma 33. There exists a constant B > such that for each N £ N, t £ Tn , £ £ X T r , £' £ Af T; j € J', 
and k £ {0, . . . , |r|}: 



P^(fcO-D^(£;OI<^> 



(214) 



where D-0j, Tfc *s as defined in Equation (69) and T)ip T j tTk is as defined in Equation (211). 
Proof. Notice by the Triangular Inequality and the Cauchy Schwartz Inequality: 

dh 



PVvWfcO-DVWfcOI< 



dx 



(<M0) 



9x 



9/z 



(0)-fl£(*r,T»(0) 



IP^r,T»«;OII 2 - (215) 



The result follows by applying Condition [I] in Assumption [2j Condition [3] in Corollary [T] Lemma [23j Lemma 
[29) and Corollary [it] ' □ 

Given these results, we can begin describing the properties satisfied by the discretized optimality function: 



Lemma 34. Let N £ N, r £ T/\r, and £ r fee defined as in Equation (157). Then there exists a constant 
L > such that, for each £i,£2,£' € <%V !r , 



Proof. Letting *+(£) = max{0, <£ T (£)} and = max{0, -* r (£)}, observe: 



(216) 



C T (£,£') = max(DJ T (£;£' -£)-*+(£), max D^r* (£;£'-£)- 7*7 (o) + US' - (217) 
I je.7, *e{o,...,|r|} J 



Employing Equation ( 74 ) 



|Cr(a,£') - Cr(Ca,0| < max||DJ T (a;£' - fc) - DJ r (6;£' - 6)| + |^(6) 

max |D^, i>T . fc (^; - ^i) -D^,i,^ (€2; - ^) | +t| *7 fe) - (Ci) | ) + 1 11^ - ^1 11^ - llC'- ^2 1|^ | - 

3&J, fce{0,...,|i-|} j 

(218) 

We show three results that taken together with the Triangular Inequality prove the desired result. First, 
by applying the reverse triangle inequality: 



2 \\x- 



(219) 
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Second, 



|DJ T (£i;£' - 6) - DJ r (6; r - 6)| = |DJ T (£i;£' - £1) - D J T (6; - 6) + DJ r (6; 6 - 6)| 

< |DJ T (ei;0-DJr(ea;0| + |DJrKi;ei)-DJ(&;Ci)| 
<9/i ( 

<£ lift -611*, 



^i(6))D0 T ,i(6;6-a) 



(220) 



where L > and we employed the linearity of DJ r , Corollary [l9j the fact that £' and £i are bounded since 
£ AV, r , the Cauchy- Schwartz Inequality, Condition [2] in Corollary 13 and Corollary 18 Notice that 
by employing an argument identical to Equation ( 220[ ) and Corollary 20 we can assume without loss of 
generality that |D^ T!i , Tji (£i;£' - £ x ) - Dip Ttj , Tk (fr,£' -£ 2 )| < i ||£i - 6 II*- Finally, notice that by applying 
(£) an d (0 are Lipschitz continuous. 



Lemma 



22 



□ 



Employing these results, we can prove that Q T (£; £') converges to £(£; £') as the discretization is increased: 
Lemma 35. There exists a constant B > such that for each N £ N, r € Tn > and £, £' G X r r : 

B 



Kr(e,0-C(e,Ol< 



2 JV' 



(221) 



where £ is as defined in Equation (|31[) and £ T is as defined in Equation (158 1. 



Proo/. Let = max{0, #(£)}, #+(£) = max{0,* T (0}, = max{0, -*(£)}, and = 

max{0, — X I' T (^)}. Notice that we can then write: 

C(£,0 = max (DJ(^' (0, max D^fe - - 7*~(£)} + II? " € Ik, (222) 
L ie.7, te[o,i] J 

and similarly for £ T (£, £')■ Employing this redefinition, notice first that by employing an argument identical 
to the one used in the proof of Lemma 25 we can show that there exists a K > such that for any positive 
integer N, t e Tn and £ e <%V iT ,: 



if 

2JV- 



(223) 



Let G {0, . . . , |r|} such that i € [T K (t), T K n\+i] for each t £ [0, 1]. Then there exists K' > such that, 



|DVi )t (£;e , -0-Dfe w (fc€'-0| < 



'' % (*(O)-|*(W0) 



0.7; 



P^tee 7 -on 



eta 



(WO) 



,(t)Vs,-s >/|| 2 

< C" (1^(0 - TK(t) (0|| a + ||D&(f; - - D0 TbW (£; £' - 0|| 2 ) 
if' 

< 



2 W ' 

(224) 

where C" > is a constant obtained after applying Corollary[8j Condition [4] in Assumption[2j and Condition 
[3] in Corollary [T] and the last inequality follows after noting that both terms can be written as the integral of 
uniformly bounded functions over an interval of length smaller than 2~ N . Thus, by the Triangular Inequality, 
Lemma [33| and Equation (224 1, we know there exists B > such that for each t € [0, 1]: 

B 



m,t (6 £' - - D^,T„ (t , & £' - 1 



< 



2 w 



(225) 
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Moreover, if t' £ argmax t£ r 01 i D-0j, t(£;£' — 0> tnen 
maxD^^-O-, max ^r^Ki^ - < - f) - D^,r. w K;f' - < |- (226) 

££[0,1J /c£ \0 : . . . , |t| J- Z 

Similarly if fc' e argmax fce{0 | r | } Dt/> Tj3 - i7 * (& £' ~ Oj then 



max n -.(£:£' - - maxD^ t (^' - < D^- , Tfc , (£; - - D^,^ - < ^. (227) 



Therefore, by Equation (226) 



max DVn — £) — max Dtp T j T , (£; £'-£)< 

ie.7,te[o,i] w ^ w ie^, fce{o I ...,|r|} Vrj ' T ^'^ 



^^te]^ ft ^-«-^«r|>^^ te ^- f) J (228) 



and similarly, by Equation (227) 



^(fc^^^ (229) 
Employing these results and Equation (74), observe that: 

Kr& O - < max{|DJ r (£; - - DJ(£; £' - 0| + |*+(0 - *+(£)| , 

+ 7 !*-(£) (230) 



max Dfrj,Ti(^-f)- max D^-,t (£;£'- 
jej, fce{o,...,|r|} jej,te[a,i] 

Finaly, applying Lemma [31] and the inequatlities above, we get our desired result. □ 

C, T is in fact strictly convex just like its infinite dimensional analogue, and its proof is similar to the proof 
of Lemma [TT] hence we omit its details. 



Lemma 36. Let N £ N, r £ Tn > ond £ £ X T V . Then the map £' t— > Cr(£)£/)> as defined in Equation (158), 
is strictly convex. 



Theorem 11 is very important since it proves that g T , as defined in Equation ( 157), is a well-defined func- 
tion. Its proof is a consequence of the well-known result that strictly-convex functions in finite-dimensional 
spaces have unique minimizers. 

Theorem 11. Let N £ N, r £ Tn, and £ £ <%V. p . TTien i/ie map £/ i— ► Cr(£;£')> as defined in Equation 



(158), has a unique minimizer. 



Employing these results we can prove the continuity of the discretized optimality function. This result is 
not strictly required in order to prove the convergence of Algorithm [2] or in order to prove that the discretized 
optimality function encodes local minimizers of the Discretized Relaxed Switched System Optimal Control 
Problem. However, this is a fundamental result from an implementation point of view, since in practice, a 
computer only produces approximate results, and continuity gives a guarantee that these approximations 
are at least valid in a neighborhood of the evaluation point. 



Lemma 37. Let N £ N and t £ Tn, then the function 8 T , as defined in Equation (157), is continuous. 
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Proof. First, we show that 9 T is upper semi-continuous. Consider a sequence {£i}ieN C X Tr converging to 
£ G X Ty r, and £' e #T,r> such that 9 T (£) = Ct{S,,^'), i-e. £' = <? T (£)) where g is defined as in Equation (157 1. 
Since r (&) < Cr (&,£') for all i G N, 

limsup0 T (&) < limsupCr(6,0 = Cr&O - OAS), (231) 

i— foo i— >oo 

which proves the upper semi-continuity of 9 T . 

Second, we show that 9 T is lower semi-continuous. Let {£i}ieN C X Tr such that T (£i) — Ct(£i>£0> i- e - 
^ = g T (£,i)- From Lemma 34 we know there exists a Lipschitz constant L > such that for each j g If, 
|Cr(f,3) " Cr(&,3)l < i ||e — Consequently, 

< (Cr(£,£) - Cr&.g)) + Ct(6,£) < " 6|U + * T (&). ( 232 ) 

Taking limits we conclude that 

T (f) <liminf0 T (&), (233) 

i— >-oo 

which proves the lower semi-continuity of 9 Tl and our desired result. □ 

Next, we prove that the local minimizers of the Discretized Relaxed Switched System Optimal Control 
Problem are in fact zeros of the discretized optimality function. 



Theorem 12. Let N G N. r G T/v, and T 6e defined as in Equation (157 1, then: 

(1) r is non-positive valued, and 

(2) If £ £ X T ,p is a local minimizer of the Discretized Relaxed Switched System Optimal Control Problem 
as in Definition^ then T (£) = 0. 

Proof. Notice Cr(£,£) = 0, therefore T (f) = mi^-^, Ct (£,£') < &-(£,£) = 0. This proves Condition [TJ 

To prove Condition [2j we begin by making several observations. Given £' € X T r and A G [0, 1], using the 
Mean Value Theorem and Corollary [19] we have that there exists s G (0, 1) and L > such that 



Jr (£ + A(£' - 0) - Jr(0 = D J T (£ + sA(£' - 0; AS' - 0) 

<ADJ T (e;?'-e)+iA 2 ||r-eil^ 



(234) 



Letting _4 T (£) = {(j,k) € J X {0, ... , |r|} | ^ T (^) = hj(z^(T)~))}, similar to the equation above, there exists 
a pair (i, fc) G _4(£ + A(£' - £)) and s G (0, 1) such that, using Corollary 



20 



< ^ + AS' - 0) " ^ (S) 

= D^ T , i , Tfc (e + sA(e , -0;A(e , -0) 

<ADV^, rfc (£;£'-£)+LA 2 ||£'-£||^. 

We prove Condition[2]by contradiction. That is we assume £ G is a local minimizer of the Discretized 
Relaxed Switched System Optimal Control Problem and 9 T (£) < and show that for each e > there exists 

I G { £ G AV, r | * T (f) < 0} n7V r ,Ar(e,e) such that J T (|) < J T (S), where Mr,x(S,e) is as defined in Equation 
(|156|, hence arriving at a contradiction. 



Before arriving at this contradiction, we make two more observations. First, notice that since £ G X p is a 
local minimizer of the Discretized Relaxed Switched System Optimal Control Problem, V E , T (£) < 0. Second, 
consider g T as defined in Equation (157), which exists by Theorem 11 and notice that since T (£) < 0, 

gAS)^S- 
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Next, observe that: 



e T (0=nuK<DJ T (C; ffr (0-0. / , m ax , , D^ TJ>fc (£; <? T (£) - + 7 *r(0 > + IMO - ClU < 0. 
I Cj.*)eJx{o.-.k|} J 

(236) 

For each A > by using Equations (233 and (236) we have: 



JA^ + A( ffT (0 - 0) - MC) < + 4A 2 LA 2 , 



(237) 



where A — max{||u||2 + 1 | u G U } and we used the fact that DJ r (£;g(£) — £) < $t(£)- H en ce for each 

■7t(£ + A(0 t (O-OWt(O<O. (238) 



\ r If) 



Similarly, for each A > by using Equations (I235J) and (236) we have: 



*r(£ + X(gA0 - 0) < *r(0 + (*r(0 - 7*r(0) A + 4 A 2 LA 2 



(239) 



where, as in Equation (237), A = max{||-u|| 2 + 1 | w € U } and we used the fact that Dip Tt j iTk (^; <?(£) — £) < 
T (f). Hence for each A £ (o,min{^^, i}): 



*r(£ + A( 5 r(0 - 0) < (1 " 7A)*r(0 < 0. 



(240) 



Summarizing, suppose £ € <Y T p is a local minimizer of the Discretized Relaxed Switched System Optimal 
Control Problem and r (O < 0. For each e > 0, by choosing any 



A G 0, min 



?r(0 1 



(241) 



4^L ' 7 ' |M0 - . 

we can construct a new point £ = (£ + A(<7 T (£) — £)) G X Tr such that £ G Af T .x{£,£) by our choice of A, 



J T (0 < Jt(£) by Equation (238), and ^ T {0 < by Equation (240). Therefore, £ is not a local minimizer 
of the Discretized Relaxed Switched System Optimal Control Problem, which is a contradiction and proves 
Condition E □ 

Finally, we prove that the Discretized Relaxed Switched System Optimal Control Problem consistently 
approximates the Switched System Optimal Control Problem: 

Theorem 13. Let {r^jigM and {£i}igN such that Tj G 71 and G X TijP for each i G N. Then 

hm |M6)- 0(6)1=0, (242) 



where 9 is as defined in Equation ( 30 ) and 9 T is as defined in Equation 



Switched System Optimal Control Problem as defined in Equation (155) is a consistent approximation of the 



Switched System Optimal Control Problem as defined in Equation (18), where consistent approximation is 
defined as in Definition^ 



157). That is, the Discretized Relaxed 



Proof. First, by Lemma [35} 



limsup0(£i) - 0Ti(£i) < hmsup - Cn (&;&-<(&)) < hmsup 



B 



0, 



(243) 



where g is as defined in Equation (30) and g T is as defined in Equation (157) 
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Now, by Condition [2] in Lemma 19 we know there exists a sequence {6'}ieN, with £| £ ^r 4 ,r for each 
ieN, such that lim^oo 6 = .9(6 ■ Then, by Lemma [35| 



limsup0 Ti (6) - 0(6) < lim sup 6,(6, 60 - C(&,<KO) 

< iimsu P (c Ti te,^) - cte,e-)) + (c(e»eo - c(&,s(0)) 



(244) 



<limsup|+Cte,^')-Cfe,5(0)- 

2 — ^OO * 



Employing Equation ( 74 ) 



C(6, CO - Cfe, 9(0) I < maxj |D Jfe; £ - 6) - D J(6; <?(£) - 6)| 

max jD^tfe^ - 6) - DV»i,tte;s(0 - 6)1 K I lie: - 6IU - 1 1 9(0 -ti\\xl (245) 



(246) 



(247) 



jej, te[o,i] 

Notice, that by applying the reverse triangle inequality: 

|lk--6IU-||ff(0-^IU|<|^-ff(0IU. 

N^Gxt notice 

|DJ&;£ - -DJ(&; <?(£)- &)| = |DJ(6; £-<?(£)) | 

5/i 



9x 



(<M6))D<M6;6' "<?(£)) 



<i|^-fl(0L. 

where L > and we employed the linearity of D J, Condition [2] in Corollary [T] and Corollary [8] Notice 



that by employing an argument identical to Equation (247), we can assume without loss of generality that 
\WjA&&-&-WiAto9{0-&\<L\\%-g(t)\\ x . Therefore: 



limsup|C(£i,$-C(fc,0(O)| <0. 



(248) 

(249) 

□ 



From Equation (244), we have limsup^^ (0 Ti (6) — 0(6)) < 0. Notice that 

limsup|0 Ti (6)- 0(6)1 > liminf |0 Ti (6)-0(6)l > 0. 

i — ^oo 2— >-oo 

Therefore combining our results, we have lim^oo |0 Ti (6) — 0(6)1 = 0- 
6.3 Convergence of the Implementable Algorithm 

In this subsection, we prove that the sequence of points generated by Algorithm [2] converge to a point that 
satisfies the optimality condition. We begin by proving that the Armijo algorithm as defined in Equation 



( 161 1 terminates after a finite number of steps. 



Lemma 38. Let a £ (0, 1) and f3 £ (0, 1). For every S > 0, there exists an M| < oo such that if r (£) < —5 



for JV € N, t e Tn 7 and £ € <-tV lP , iften /z T (6) < -Mj> where T is as defined in Equation (157) and /i T is as 
defined in Equation (161). 



Proof. Given (J £ X and A £ [0, 1], using the Mean Value Theorem and Corollary 19 we have that there 
exists s £ (0, 1) such that 



Jr{i + A(6 - 0) - MS) = D^(C + 8\(? ~ f); A(f - £)) 

<ADj r (66-0+ iA2 lie-eil; 



(250) 
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Letting A T (0 = € J X {0, . . . , |r|} | * r (£) = ip T ,j,Tt(0}> then there exists a pair e A T (S, + \{£- 

£)) and s £ (0, 1) such that, using Corollary 



11 



<VrJ,r 4 (f + A(e'-0)-V'r J -,n.(0 

= D^, T( (e + sA(^-0;A(e'-0) 
< AD^(e;e'-0+ LA2 |l^-ell^- 



Now let us assume that ^ T (0 < 0, and consider g T as defined in Equation (157). Then 

{ (j,i)Gjx{0,...,\r\} J 



and using Equation ([250]), 

Jt(H + P k (9r(0 - £)) - Jr(f) - a/J k 0r(O < "(1 " + ^Lf?\ 



where A = max {||u||2 + 1 | u G {/}. Hence, for each fc g N such that < 
Similarly, using Equations (12511) and (12521) , 



we have that 



*r(C + /3 fe ( 5 (£) - 0) - *r(0 + 0*(7*r(O - a0 T (f)) < -<^ fe + 4A 2 L/? 2fe , 
hence for each fceN such that /3 fe < min j , i j we have that 

*r (£ + /3 fc (ff(0 - 0) - «/?^r(0 < (1 - < 0. 

If M£) > then 

max D^ n ({ ;jr (fl - < MO < -6, 

(j,t)ejx{0,...,\r\} 



thus, from Equation ([251]), 

M* + /3 fc (.9r(£) - 0) - M0 - «/? fc T (O < -(1 - + AA 2 L(3 2k . 

Hence, for each k € N such that /3 fe < we have that 



Finally, let 



M* + /3*(ft-(0 - 0) - M0 < af3 k 9 T (0- 
'(l-a)S 



Rig = 1 + max { lo 



AA 2 L 



then from Equations (254), (256), and (259), we get that /i T (0 < Mj- as desired. 



(251) 



(252) 

(253) 

(254) 
(255) 

(256) 
(257) 

(258) 

(259) 

(260) 

□ 



The proof of the following corollary follows directly from the estimates of M| in the proof of Lemma 38 



Corollary 21. Let a G (0, 1) and (3 G (0, 1). There exists a 5q > and C > suc/i that if S € (0, o"o) and 
0t(£) ^ — ^ / or JV G N, r G T/v. i^rf £ € ^r.p; i/ien /i T (0 < 1 + logo(C<5), where 8 T is as defined in Equation 
( 157[ ) and /i T is as defined in Equation (161). 

Next, we prove a bound between the discretized trajectory for a point in the discretized relaxed opti- 
mization space and the discretized trajectory for the same point after projection by pjy that we use in a later 
argument. 
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Lemma 39. Consider pn defined as in Equation (361 and o~n defined as in Equation (162). There exists 

(261) 



K > such that for each N ,N 6 N, r E Tn , £, = (u, d) E A? r>T , and t E [0, 1] : 

N / -, x N 



\\^ N (cApn(0) - M0\\ 2 < k ^) (hciibv + 1) + Q) 

where 4> T j is as defined in Equation (152). 

Proof. We prove this argument for t = 1, but the argument follows identically for all t E [0, 1]. Using the 
triangular inequality we have that 

\\<Pv n (0,i(pn(0) - M0\\ 2 < - Mpn(0)\\ 2 + \\Mpn(0) - 0i(O|| a + 

+ ||0i(O-^r,i(O|| 2 - ( 26 2) 

Thus, by Theorem [6] and Lemma 23 there exists K\, K 2 , and .K3 such that 



|<^(0,i(M6) - ^.iCOlla < *i (^J (IKIbv + 1) + ^ + 



2^o ' 



(263) 

□ 



hence the result follows after organizing the constants and noting that 2 2 < 2 N for each N E N. 
Using this previous lemma, we can prove that v T is eventually finite for all £ such that < 0. 

Lemma 40. Let Nq E N, tq E Tn oj cind £ E X T ,r- If < then for each r\ E N t/iere exists a finite 
N > iVo smc/i i/iaf; zv N ({) (£, N + ij) is finite. 



Proof. Recall v T , as defined in Equation |163j), is infinity only when the optimization problem it solves is 
not feasible. To simplify our notation, let £' E X CTN ^ r defined by £' = £ + /3 M "« (5)(?) (g (JN (^)(£) - £). Then, 



using Lemma 39 for k E N, k E [N, N + 77] 

■Wo MO) - ^(o(e') < lk U -L)* (He'llBv + 1) + Q) 



A' 



(264) 



llC||w + 2) 



Also, from Theorem 13 we know that for N large enough, 

1 



20(£)><W)(O. 

Thus, given 5 > §0(0) there exists N* E N such that, for each N > N* and fee [AT, JV + 77], 

■W)MO) - -W)(£') < -«^*(0 



(265) 



and at the same time 



<-S)9^ (0 (0. 
a/3 w < (1 - uj)a(3 M i < (1 - w)a/3^«(«, 
Similarly, given A(0 ~{(i,*) € .7x [0,1] | * T (0 = ^rj,t(0}. let 0", *) e 4r w ({<)(?)• Thus, for N > 



(266) 
(267) 



where Mi is as in Lemma 38 



N*, k E [N, N + 77], and using Lemma 39 



1 



< LX 



AT 



(lirilw + 2) 



(268) 
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Therefore, for N > N* , if * (Tiv(e) (^) < 0, then by Equations ( [266] ), p68| , and the inequalities from the 
computation of /J, T {£,)i 

~ J* N (0(0 < - ap N )e aN{ y(0, (269) 

(PfcCO) < («/^~ (?) - 5^)^(0(0 < 0, (270) 

which together with Equation ([267 ) implies that the feasible set is not empty. Similarly, if ty aN (£)(£) > 0, 
by Equation p68| , 

*<t»«o(p*(0) -*<r W (e)(0 < (o^ (0 - ap N )e aN ^(0, (271) 

as desired. 

Hence for all N > N* the feasible sets of the optimization problems associated with v aN (£) are n °t empty, 
and therefore v<t n (£) (£, N + -q) < oo. □ 

In fact, the discretization precision constructed by Algorithm [2] increases arbitrarily. 

Lemma 41. Let {Ni}i^, {r^ligN, and {b}ieN be the sequences generated by Algorithm^ Then Ni — > oo 
as i — » oo. 

Proof. Suppose that Ni < N* for all i€N. Then, by definition of Algorithm [2j there exists io £ N such that 
< -A2~ xNi < -A2-* N ' and & +1 = r Ti (&) for each i > i , where T T is defined in Equation ( [164] . 
Moreover, by definition of z/ T we have that if there exists ii>io such that \t Tii (£i x ) < 0, then fy T( < 



;-!8 



for each i > ii. Let us assume that there exists i\ > zo such that <t Ti < 0, then, using Lemma 

J n+1 (b + i) Jn(Zi) < (*P^ - aF^M^Ofo) 

for each i > ij, where J' = A2 _xAr . But this implies that J Ti (£i) — >• -co as i — >■ oo, which is a contradiction 
since /io ; arL d therefore J Ti , is lower bounded. 

The argument is completely analogous in the case where the sequence is perpetually infeasible. Indeed, 



suppose that ^ n (b) > for each i>io, then by Lemma 38 



for each i > i , where <5' = A2~ xAr . But again this implies that \& Ti (&) — > — oo as i — > oo, which is a 
contradiction since we had assumed that \& Ti (£i) > 0. □ 

Next, we prove that if Algorithm [2] find a feasible point, then every point generated afterwards remains 
feasible. 

Lemma 42. Let {Ni}i^, {ri}^}, and {£i}ieN be the sequences generated by Algorithm^ Then there exists 
io G N such that, if ^ Tio (£,io) — ®> then ^(Ci) < and ^ Ti (Ci) ^ f or each i > io, where ^ T is as defined in 
Equation ( |151[ ). 

Proof. Let I C N be a subsequence defined by 

X = |* e N | Tj (6) < and u Ti N + rf) < ooj . (274) 

Note that, by definition of Algorithm [2j ^(b+i) = f° r each i ^ X. Now, for each i £ X such that 

*n(£i) < 0> by definition of v T in Equation (163) together with Corollary [2lj 



2x Ni (275) 



< -Loa(3C 



( A 



2 
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(276) 



where C > 0. By Lemma 25 and the fact that Ni+i > iVj, we have that 

Hence, if < for ii € N such that TV^ is large enough, then \I/(£i) < for each i > i\. 

Moreover, from Equation (276) we get that for each N > Ni and each r 6 Tat, 

; C ( 277 ) 

1 / 25 



Thus, if fy Ti2 (£i 3 ) < for i 2 € N such that iV i2 is large enough, then 5 , T (^i 2 ) < for each r G T/v such that 
N > Ni. But note that this is exactly the case when i 2 + k £ I for fc e {1, . . . , n}, thus we can conclude 
that ^T i2+k (£,i 2 +k) < 0. Also note that the case of i G I is trivially satisfied by the definition of v r . 

Finally, by setting iq = maxjii,^} we get the desired result. □ 

Next, we prove 9 T converges to zero. 

Lemma 43. Let {iVj}j 6 N ; {Ti}»6N; and {£i}igN be the sequences generated by Algorithm^ Then 9 Ti (^i) — > 
as i — > oo, where 9 T is as defined in Equation (157). 

Proof. Let us suppose that limj_ J . tx) 9 Ti (£i) 0- Then there exists S > such that 

liminf Ti (&) < -4(5, (278) 

i— >oo 

and hence, using Theorem [13] and Lemma |4l"| there exists an infinite subsequence KcN defined by 

JC = {i E N | 9 Ti (£i) < -28 and 0(&) < -5}. (279) 
Let us define a second subsequence IcNby 

i G N | 6> T . (&) < and i/ T< + 77) < oo J . (280) 

Note that by the construction of the subsequence IC, together with Lemma [40} we get that Kf)I is an infinite 
set. 

Now we analyze Algorithm [2] by considering the behavior of each step as a function of its membership to 
each subsequence. First, for each i ^ I, £,- + x = thus J(£,+i) = J(£i) and \l/(£ i+ i) = 4'(Ci)- Second, let 
i e 1 such that * Ti te') < °j then 



J, 



r i+1 (&+i) ~ JtM < - ap^ m+1l) ) 9 T M) 



2^v7 (281) 



where C > and the last inequality follows from Corollary 21 Recall that iVj+i > iVf, thus using Lemmas 
l24l and [41] we have that 

2 ' V ^ 7 (282) 

1 / 2_B x 

- 2 2 xNi \ 2(l-2 X )iVi 
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and since x G (0, |), we get that for N t large enough J(£ i+ i) < J(£i). Similarly, if \|/ r . (^) > then 



+1J 



i 



2B 



22 X N t l 2 (1-2 X )JV 4 



(283) 



thus for TVj large enough, < Third, let « G /CnX such that * n te) < 0, then, by Lemma 38 



< -2uaf3 M ™6, 
thus, by Lemmas [24] and [41] for TV^ large enough, 



Similarly, if \1/ T . (^) > 0, using the same argument and Lemma 25 for iVj large enough, 



(284) 



(285) 



(286) 



Now let us assume that there exists io £ N such that iVi is large enough and \& T( (£j ) < 0. Then by 
Lemma 42 we get that ^ Ti (£,i) < for each i > Iq. But as shown above, either i ^ K P\I and J(£j+i) < J(£j) 



or i € /C n I and Equation ( |285| ) is satisfied, and since JC n I is an infinite set we get that J(£i) — > — oo as 
i — > oo, which is a contradiction as J is lower bounded. 

On the other hand, if we assume that *f> Ti > for each i€N, then either i ^ K.D1 and < ^(fi) 

or i G /C l~l 1 and Equation (286) is satisfied, thus implying that — > — oo as i — > oo. But this is a 
contradiction since, by Lemma 25 this would imply that ^ Ti (Ci) ~ * ~°° as i — > oo. 

Finally, both contradictions imply that 9 n (£i) — ► as i — > oo as desired. □ 

In conclusion, we can prove that the sequence of points generated by Algorithm [2] converges to a point 
that is a zero of 9 or a point that satisfies our optimality condition. 

Theorem 14. Let {Ni}^^, {Ti}i e f), and {£i}igN be the sequences generated by Algorithm^ then 

lim = 0, (287) 



where 9 is as defined in Equation ( 30 ) . 



Proof. This result follows immediately from Lemma 43 after noticing that the Discretizcd Relaxed Switched 
System Optimal Control Problem is a consistent approximation of the Switched System Optimal Control 
Problem, as is proven in Theorem |13[ and applying Theorem |10[ □ 



7 Examples 

In this section, we apply Algorithm [2] to calculate an optimal control for four examples. Before describing 
each example, we begin by describing the numerical implementation of Algorithm [2] First, observe that the 
analysis presented thus far does not require that the initial and final times of the trajectory of switched system 
be fixed to and 1, respectively. Instead, the initial and final times of the trajectory of the switched system 
are treated as fixed parameters to and tt, respectively. Second, we employ a MATLAB implementation of 
LSSOL from TOMLAB in order to compute the optimality function at each iteration of the algorithm since 
it is a quadratic program [21 . Third, for each example we employ a stopping criterion that terminates 
Algorithm [2] if 9 T becomes too large. Each of these stopping criteria is described when we describe each 
example. Next, for the sake of comparison we compare the performance of Algorithm [2] on each of the 
examples to a traditional Mixed Integer Program (MIP). To perform this comparison, we employ a TOMLAB 
implementation of a MIP described in |12] which mixes branch and bound steps with sequential quadratic 
programming steps. Finally, all of our comparisons are performed on an Intel Xeon, 6 core, 3.47 GHz, 100 
GB RAM machine. 
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Example 


Mode 1 


Mode 2 


Mode 3 


LQR 


x(t) = Ax(t) + 


" 0.9801 " 
-0.1987 



u(t) 


= Ac(f) + 


" 0.1743 " 
0.8601 
-0.4794 


tt(t) 


x(t) = Ax(t) + 


"0.0952" 

0.4699 

0.8776^ 


u(t) 


Tank 




±(t)- 2 -VMt) 


N/A 


Quadrotor 


*(*) = 


S ^&{u(t) + Mg) 
Soo^m ( u ( t ) + Mg)-g 





x(t) = 


gsmxz{t) 
gcosx 3 (t) - g 

-Lu(t) 
L 7 




x(t) = 


.gsinx 3 (t) 
gcosx3(t) — g 

Lu(t) 
I 




Needle 


i(t) = 


sin (x 5 (t))zti(t) 
— cos (xs(t)) sin (2:4 (£)) u\ (t) 
COS (X4(t)) cos (xc,(t))ui{t) 
kcos (xg(i)) sec (x5(i)Jui(t) 
Ksin (x 6 (t))ui(t) 
— kcos tan (x§(t))U\(t) 




x(t) = 








u 2 (t) 




N/A 



Table 1: The dynamics of each of the modes of the switched system examples considered in Section [jj The 
parameters employed during the application of Algorithm [2] are defined explicitly in Section [71 



Example 


L(x(t),u(t),t) 


*(»«>(*/)) 


[/ 


7 


Q 


|8 


a 





A 


X 




to 




LQR 


0.01 • (ii(t)) 2 




\xi(t f ) - 1] 
x 2 (t f ) - 1 
[x 3 {t f ) - lj 


2 
2 


u(t) € [-20,20] 


1 


0.1 


0.87 


0.005 


0.72 


10~ 4 


l 

4 


10" 6 





2 


Tank 


2- (x 2 {t)-3) 2 





N/A 


100 


0.01 


0.75 


0.005 


0.72 


io- 4 


1 
4 


10~ 6 





10 


Quadrotor 








/5-(s; 1 (t / )-6 

/5-M*/)-i 


)" 

) 


2 
2 


6 [0, 10- 3 ] 


10 


0.01 


0.80 


5 x 10" 4 


0.72 


io- 4 


1 
4 


IO" 6 





7.5 


Needle 


HI MIC 




" :ri(*/) +2 " 
x 2 (t/)-3.5 
x 3 (t f )-W 


2 
2 


m{t) e [0,5] 
«2(t) e [=* ,f] 


100 


0.002 


0.72 


0.001 


0.71 


10" 4 


1 
4 


0.05 





8 



Table 2: The algorithmic parameters and cost function used for each of the examples during the implemen- 
tation of Algorithm [2| 



7.1 Constrained Switched Linear Quadratic Regulator (LQR) 

Switched Linear Quadratic Regulator (LQR) examples have been used to illustrate the utility of a variety of 
proposed optimal control algorithms [TUHH]. We consider an LQR system in three dimensions, with three 
discrete modes, and a single continuous input. The dynamics in each mode are as described in Table [T] where: 



.4 



1.0979 -0.0105 0.0167 
-0.0105 1.0481 0.0825 
0.0167 0.0825 1.1540 



(288) 



The system matrix is purposefully chosen to have three unstable eigenvalues and the control matrix in each 
mode is only able to control along single dimension. Hence, while the system and control matrix in each 
mode is not a stabilizable pair, the system and all the control matrices taken together simultaneously is 
stabilizable and is expected to appropriately switch between the modes to reduce the cost. The objective of 
the optimization is to have the trajectory of the system at time tf be at (1, 1, 1) while minimizing the input 
required to achieve this task. This objective is reflected in the chosen cost function which is described in 
Table H 
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Example 


Initial Continuous 
Input, Vt 6 [t , if] 


Initial Discrete 
Input, Vt 6 [to, t/] 


Algorithm |2| 
Computation Time 


Algorithm |2| 
Final Cost 


MIP 

Computation Time 


MIP 
Final Cost 


LQR 


u(t) = 


rf(t) = 


T 






9.827[s] 


1.23 x KT 3 


753.0[s] 


1.89 x 10- 3 


Tank 


N/A 


d(i) = 


"1" 






32.38[s] 


4.829 


119700[s] 


4.828 


Quadrotor 


u(t) = 5 x 10~ 4 


d(t) = 


1.33 
).34 
).33 




8.350[s] 


0.128 


2783 [s] 


0.165 


Needle 


«(i)= J 


d(i) = 


0.5 

0.5 




62.76[s] 


0.302 


did not converge 


did not converge 



Table 3: The initialization parameters used for each of the examples during the implementation of Algorithm 
ffl and the MIP described in [T^] , and the computation time and the result for each of the examples as a 
result of the application of Algorithm [2] and the MIP described in [T3] . 



Algorithm [2] and the MIP are initialized at xq = (0, 0, 0) with continuous and discrete inputs as described 
in Table [3] with 16 equally spaced samples in time. Algorithm [2] took 11 iterations, ended with 48 time sam- 
ples, and terminated after the optimality condition was bigger than — 10 -2 . The result of both optimization 
procedures is illustrated in Figure [T] The computation time and final cost of both algorithms can be found 
in Table [3J Notice that Algorithm^] is able to compute a lower cost continuous and discrete input when 
compared to the MIP and is able to do it more than 75 times faster. 

7.2 Double Tank System 

To illustrate the performance of Algorithm [2] when there is no continuous input present, we consider a 
double-tank example. The two states of the system correspond to the fluid levels of an upper and lower 
tank. The output of the upper tank flows into the lower tank, the output of the lower tank exits the system, 
and the flow into the upper tank is restricted to either 1 or 2. The dynamics in each mode are then derived 
using Toricelli's Law and are describe in Table [l] The objective of the optimization is to have the fluid level 
in the lower tank track 3 and this is reflected in the chosen cost function described in Table [51 

Algorithm [2] and the MIP are initialized at Xq — (0, 0) with a discrete input described in Table [3] 
with 128 equally spaced samples in time. Algorithm [2] took 67 iterations, ended with 256 time samples, 
and terminated after the optimality condition was bigger than — 10~ 2 . The result of both optimization 
procedures is illustrated in Figure [2] The computation time and final cost of both algorithms can be found 
in Table [3] Notice that Algorithm |2] is able to compute a comparable cost discrete input compared to the 
MIP and is able to do it nearly 3700 times faster. 

7.3 Quadrotor Helicopter Control 

Next, we consider the optimal control of a quadrotor helicopter in 2D using a model described in [IB]. The 
evolution of the quadrotor can be defined with respect to a fixed 2D reference frame using six dimensions 
where the first three dimensions represent the position along a horizontal axis, the position along the vertical 
axis and the roll angle of the helicopter, respectively, and the last three dimensions represent the time 
derivative of the first three dimensions. We model the dynamics as a three mode switched system (the first 
mode describes the dynamics of going up, the second mode describes the dynamics of moving to the left, 
and the third mode describes the dynamics of moving to the right) with a single input as described in Table 
[T] where L = 0.3050 meters, M = 1.3000 kilograms, I = 0.0605 kilogram meters squared, and g — 9.8000 
meters per second squared. The objective of the optimization is to have the trajectory of the system at time 
tf be at position (6, 1) with a zero roll angle while minimizing the input required to achieve this task. This 
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(a) MIP Final Result (b) Algorithm [2] Final Result 

Figure 1: Optimal trajectories for each of the considered optimization algorithms where the point (1, 1, 1) is 
drawn in green, and where the trajectory is drawn in blue when in mode 1, in purple when in mode 2, and 
in red when in mode 3. 



3.6 r 3.6 r 




Time [s] Time [s] 



(a) MIP Final Result (b) Algorithm [2] Final Result 

Figure 2: Optimal trajectories for each of the considered optimization algorithms where X\{t) is drawn using 
points and x% (t) is drawn using stars and where each state trajectory is drawn in blue when in mode 1 and 
in purple when in mode 2. 
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(a) MIP Final Result 



(b) Algorithm p\ Final Result 



Figure 3: Optimal trajectories for each of the considered optimization algorithms where the point (6, 1) is 
drawn in green, where the trajectory is drawn in blue when in mode 1, in purple when in mode 2, and in red 
when in mode 3, and where the quadrotor is drawn in black and the normal direction to the frame is drawn 
in gray. 

objective is reflected in the chosen cost function which is described in Table [2] A state constraint is added 
to the optimization to ensure that the quadrotor remains above ground. 

Algorithm [2] and the MIP are initialized at position (0, 1) with a zero roll angle, with zero velocity, with 
continuous and discrete inputs as described in Table[3j and with 64 equally spaced samples in time. Algorithm 
[2] took 31 iterations, ended with 192 time samples, and terminated after the optimality condition was bigger 
than — 10~ 4 . The result of both optimization procedures is illustrated in Figure [3] The computation time 
and final cost of both algorithms can be found in Table [3j Notice that Algorithm [2] is able to compute a 
lower cost continuous and discrete input when compared to the MIP and is able to do it more than 333 times 
faster. 

7.4 Bevel-Tip Flexible Needle 

Bevel-tip flexible needles are asymmetric needles that move along curved trajectories when a forward pushing 
force is applied. The 3D dynamics of such needles has been described in [23] and the path planning in the 
presence of obstacles has been heuristically considered in [8]. The evolution of the needle can be defined 
using six dimensions where the first three dimensions represent the position of the needle relative to the 
point of entry and the last three dimensions represent the yaw, pitch and roll of the needle relative to the 
plane, respectively. As suggested by [8], the dynamics of the needle are naturally modeled as a two mode 
(the first mode describes the dynamics of going forward while the second mode describes the dynamics of 
the needle turning) switched system as described in Table [I] with two continuous inputs: u\ representing 
the insertion speed and 112 representing the rotation speed of the needle and where k is the curvature of the 
needle and is equal to .22 inverse centimeters. The objective of the optimization is to have the trajectory of 
the system at time tf be at position (—2,3.5, 10) while minimizing the input required to achieve this task. 
This objective is reflected in the chosen cost function which is described in Table [2] A state constraint is 
added to the optimization to ensure that the needle remains outside of three spherical obstacles centered at 
(0 , , 5), (1 , 3 , 7), and (-2 , , 10) all with radius 2. 

Algorithm [2] and the MIP are initialized at position (0, 0, 0) with continuous and discrete input described 
in Table [3] with 64 equally spaced samples in time. Algorithm [2] took 103 iterations, ended with 64 time 
samples, and terminated after the optimality condition was bigger than — 10~ 3 . The computation time and 
final cost of both algorithms can be found in Table [3| The MIP was unable to find any solution. The result 
of Algorithm [2] is illustrated in Figure |4| 
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8 



8 



(a) Algorithm pi Final Result (b) Algorithm pi Final Discrete Input 



Figure 4: The optimal trajectory and discrete inputs drawn in cyan generated by Algorithm [2j where the 
point (—2, 3.5, 10) is drawn in green and obstacles are drawn in grey. 

8 Conclusion 

In this paper, we devise a first order numerical optimal control algorithm for the optimal control of con- 
strained nonlinear switched systems. The algorithm works by first relaxing the discrete-valued input, per- 
forming traditional optimal control, and projecting the computed relaxed discrete- valued input by employing 
a projection constructed by an extension to the classical Chattering Lemma. We prove that the sequence of 
points constructed by recursive application of our algorithm converge to a point that satisfies a necessary con- 
dition for optimality of the Switched System Optimal Control Problem. We then devise an implementable 
algorithm that operates over finite dimensional subspaces of the optimization spaces. We prove the con- 
vergence of the sequence of points constructed by recursive application of our computationally tractable 
algorithm to a point that satisfies a necessary condition for optimality of the Switched System Optimal 
Control Problem. 
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